2023-01-25T05:09:53.699 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T05:09:53.701 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814 --verbose --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814/orig.config.yaml 2023-01-25T05:09:53.745 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T05:09:53.833 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814 branch: wip-yuri-testing-2023-01-23-0926 description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} email: yweinstein@radhat.com first_in_suite: false job_id: '7136814' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async osd_object_clean_region_max_num_intervals: 1000 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd inject bad map crc probability: 0.1 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 2 - rados: balance_reads: true clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T05:09:53.833 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T05:09:53.834 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T05:09:53.834 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T05:09:53.836 INFO:teuthology.task.internal:Checking packages... 2023-01-25T05:09:53.865 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T05:09:53.865 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T05:09:53.865 INFO:teuthology.packaging:ref: None 2023-01-25T05:09:53.866 INFO:teuthology.packaging:tag: None 2023-01-25T05:09:53.866 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T05:09:53.866 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:09:53.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T05:09:54.022 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959.gb0d25925 2023-01-25T05:09:54.022 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T05:09:54.029 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T05:09:54.029 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T05:09:54.050 INFO:teuthology.task.internal:Saving configuration 2023-01-25T05:09:54.070 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T05:09:54.077 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T05:09:54.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 05:05:16.891548', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T05:09:54.129 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi085.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 05:05:16.893418', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T05:09:54.129 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T05:09:54.136 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T05:09:54.136 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T05:09:54.136 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T05:09:54.209 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe568573670>, signals=[15]) 2023-01-25T05:09:54.210 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T05:09:54.216 INFO:teuthology.task.internal:Opening connections... 2023-01-25T05:09:54.217 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:09:54.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:09:54.289 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:09:54.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:09:54.358 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T05:09:54.365 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-01-25T05:09:54.387 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-01-25T05:09:54.387 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-01-25T05:09:54.447 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T05:09:54.447 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T05:09:54.448 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-01-25T05:09:54.448 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-01-25T05:09:54.448 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-01-25T05:09:54.448 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-01-25T05:09:54.448 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T05:09:54.449 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-01-25T05:09:54.449 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T05:09:54.449 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T05:09:54.449 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T05:09:54.449 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T05:09:54.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:09:54.450 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T05:09:54.450 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T05:09:54.450 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T05:09:54.450 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T05:09:54.451 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-25T05:09:54.471 DEBUG:teuthology.orchestra.run.smithi085:> uname -m 2023-01-25T05:09:54.491 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2023-01-25T05:09:54.491 DEBUG:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2023-01-25T05:09:54.549 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T05:09:54.549 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T05:09:54.549 INFO:teuthology.orchestra.run.smithi085.stdout:ID="rhel" 2023-01-25T05:09:54.550 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="fedora" 2023-01-25T05:09:54.550 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8.6" 2023-01-25T05:09:54.550 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2023-01-25T05:09:54.550 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T05:09:54.550 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2023-01-25T05:09:54.551 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T05:09:54.551 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T05:09:54.551 INFO:teuthology.orchestra.run.smithi085.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T05:09:54.551 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T05:09:54.551 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:09:54.552 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T05:09:54.552 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T05:09:54.552 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T05:09:54.552 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T05:09:54.553 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2023-01-25T05:09:54.571 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T05:09:54.581 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T05:09:54.589 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T05:09:54.590 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-01-25T05:09:54.593 DEBUG:teuthology.orchestra.run.smithi085:> test '!' -e /home/ubuntu/cephtest 2023-01-25T05:09:54.611 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T05:09:54.617 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T05:09:54.617 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-01-25T05:09:54.653 DEBUG:teuthology.orchestra.run.smithi085:> test -z $(ls -A /var/lib/ceph) 2023-01-25T05:09:54.675 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T05:09:54.741 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T05:09:54.756 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T05:09:54.757 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T05:09:54.757 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T05:09:54.758 DEBUG:teuthology.orchestra.run.smithi085:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T05:09:54.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:09:54.777 DEBUG:teuthology.orchestra.run.smithi085:> uname -r 2023-01-25T05:09:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:09:54.778 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-01-25T05:09:54.836 INFO:teuthology.orchestra.run.smithi085.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T05:09:54.837 INFO:teuthology.task.kernel:Running kernel on smithi085: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:09:54.837 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum install -y kernel 2023-01-25T05:09:54.838 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T05:09:54.839 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:09:54.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-01-25T05:09:56.094 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:09:56.094 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T05:09:56.173 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:09:56.173 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2023-01-25T05:09:57.626 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-01-25T05:09:57.642 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-01-25T05:09:58.652 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2023-01-25T05:09:58.659 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2023-01-25T05:10:01.644 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 354 kB/s | 24 kB 00:00 2023-01-25T05:10:01.960 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 129 kB/s | 24 kB 00:00 2023-01-25T05:10:02.289 INFO:teuthology.orchestra.run.smithi085.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T05:10:02.344 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:10:02.345 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:10:02.345 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:10:02.507 DEBUG:teuthology.orchestra.run.smithi085:> echo no | sudo yum reinstall kernel || true 2023-01-25T05:10:02.620 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T05:10:02.677 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:10:02.678 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:10:02.679 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:10:02.819 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2023-01-25T05:10:02.841 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:10:02.841 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2023-01-25T05:10:03.100 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 05:10:01 AM UTC. 2023-01-25T05:10:03.150 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:10:03.174 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:10:03.175 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T05:10:03.198 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2023-01-25T05:10:03.243 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum reinstall -y kernel || true 2023-01-25T05:10:03.447 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 05:10:01 AM UTC. 2023-01-25T05:10:03.492 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:10:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-01-25T05:10:03.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2023-01-25T05:10:03.573 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:10:03.573 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2023-01-25T05:10:03.823 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Jan 2023 05:10:01 AM UTC. 2023-01-25T05:10:03.862 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:10:03.920 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2023-01-25T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T05:10:03.966 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q kernel --last | head -n 1 2023-01-25T05:10:04.045 INFO:teuthology.orchestra.run.smithi085.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T05:10:04.047 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:10:04.047 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T05:10:04.047 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T05:10:04.048 DEBUG:teuthology.orchestra.run.smithi085:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T05:10:04.073 INFO:teuthology.orchestra.run.smithi085.stdout:ttyS1 2023-01-25T05:10:04.092 DEBUG:teuthology.parallel:result is None 2023-01-25T05:10:04.185 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Jan 2023 05:10:01 AM UTC. 2023-01-25T05:10:04.230 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:10:04.260 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-01-25T05:10:04.306 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel --last | head -n 1 2023-01-25T05:10:04.388 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T05:10:04.390 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:10:04.390 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T05:10:04.390 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T05:10:04.391 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T05:10:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-01-25T05:10:04.443 DEBUG:teuthology.parallel:result is None 2023-01-25T05:10:04.444 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T05:10:04.457 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T05:10:04.458 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T05:10:04.487 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T05:10:04.505 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T05:10:04.515 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T05:10:04.529 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T05:10:04.530 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T05:10:04.545 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T05:10:04.577 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T05:10:04.583 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T05:10:04.584 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T05:10:04.604 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T05:10:04.641 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:10:04.648 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:10:04.652 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:10:04.659 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:10:04.660 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T05:10:04.668 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T05:10:04.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T05:10:04.697 DEBUG:teuthology.orchestra.run.smithi085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T05:10:04.728 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T05:10:04.741 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T05:10:04.741 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T05:10:04.765 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T05:10:04.786 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:10:04.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:10:04.911 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:10:04.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:10:05.038 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:10:05.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T05:10:05.106 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:10:05.146 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:10:05.197 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:10:05.278 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:10:05.322 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:10:05.323 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T05:10:05.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-01-25T05:10:05.394 DEBUG:teuthology.orchestra.run.smithi085:> sudo service rsyslog restart 2023-01-25T05:10:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T05:10:05.490 INFO:teuthology.orchestra.run.smithi085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T05:10:05.966 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T05:10:05.974 INFO:teuthology.task.internal:Starting timer... 2023-01-25T05:10:05.974 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T05:10:05.986 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T05:10:05.996 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T05:10:05.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-01-25T05:10:06.059 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-01-25T05:10:06.061 DEBUG:teuthology.orchestra.run.smithi085:> sudo service auditd rotate 2023-01-25T05:10:06.115 INFO:teuthology.orchestra.run.smithi085.stdout:Rotating logs: 2023-01-25T05:10:06.117 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T05:10:06.117 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-01-25T05:10:06.136 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-01-25T05:10:06.136 DEBUG:teuthology.orchestra.run.smithi085:> /usr/sbin/getenforce 2023-01-25T05:10:06.155 INFO:teuthology.orchestra.run.smithi085.stdout:Permissive 2023-01-25T05:10:06.156 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi085.front.sepia.ceph.com': 'permissive'} 2023-01-25T05:10:06.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T05:10:06.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:10:06.205 DEBUG:teuthology.orchestra.run.smithi085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T05:10:06.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:10:06.232 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T05:10:06.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-01-25T05:10:06.276 DEBUG:teuthology.orchestra.run.smithi085:> sudo /usr/sbin/setenforce permissive 2023-01-25T05:10:06.302 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T05:10:06.314 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-25T05:10:06.319 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-25T05:10:06.478 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T05:10:06.495 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T05:10:06.497 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi085.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T05:16:55.499 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi085.front.sepia.ceph.com')] 2023-01-25T05:16:55.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-25T05:16:55.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:16:55.571 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-01-25T05:16:55.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2023-01-25T05:16:55.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:16:55.707 DEBUG:teuthology.orchestra.run.smithi085:> true 2023-01-25T05:16:55.777 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T05:16:55.792 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T05:16:55.793 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T05:16:55.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T05:16:55.796 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T05:16:55.797 DEBUG:teuthology.orchestra.run.smithi085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T05:16:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T05:16:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T05:16:55.849 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T05:16:55.863 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T05:16:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-01-25T05:16:55.926 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-01-25T05:16:55.936 INFO:teuthology.orchestra.run.smithi085.stderr:sudo: ntpd: command not found 2023-01-25T05:16:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T05:16:55.947 INFO:teuthology.orchestra.run.smithi085.stdout:506 Cannot talk to daemon 2023-01-25T05:16:55.960 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T05:16:55.962 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T05:16:55.976 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T05:16:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-01-25T05:16:56.034 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T05:16:56.034 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-01-25T05:16:56.034 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.034 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.035 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.035 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.052 INFO:teuthology.orchestra.run.smithi085.stderr:bash: ntpq: command not found 2023-01-25T05:16:56.057 INFO:teuthology.orchestra.run.smithi085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T05:16:56.057 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================== 2023-01-25T05:16:56.058 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.058 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.058 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.059 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:16:56.059 INFO:teuthology.run_tasks:Running task install... 2023-01-25T05:16:56.070 DEBUG:teuthology.task.install:project ceph 2023-01-25T05:16:56.071 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T05:16:56.071 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T05:16:56.072 INFO:teuthology.task.install:Using flavor: default 2023-01-25T05:16:56.082 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T05:16:56.082 INFO:teuthology.task.install:extra packages: [] 2023-01-25T05:16:56.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T05:16:56.126 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T05:16:56.128 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:16:56.129 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2023-01-25T05:16:56.129 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T05:16:56.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T05:16:56.171 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2023-01-25T05:16:56.173 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:16:56.173 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T05:16:56.174 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2023-01-25T05:16:56.210 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T05:16:56.211 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:16:56.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:16:56.255 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2023-01-25T05:16:56.256 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:16:56.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:16:56.363 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T05:16:56.363 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T05:16:56.408 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T05:16:56.408 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T05:16:56.479 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T05:16:56.480 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:16:56.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T05:16:56.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T05:16:56.508 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T05:16:56.520 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T05:16:56.520 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:16:56.521 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T05:16:56.548 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T05:16:56.548 DEBUG:teuthology.orchestra.run.smithi085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T05:16:56.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T05:16:56.616 DEBUG:teuthology.orchestra.run.smithi085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T05:16:56.655 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T05:16:56.693 DEBUG:teuthology.orchestra.run.smithi085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T05:16:56.722 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-01-25T05:16:56.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-01-25T05:16:56.759 INFO:teuthology.orchestra.run.smithi085.stdout:check_obsoletes = 1 2023-01-25T05:16:56.761 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum clean all 2023-01-25T05:16:57.117 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:16:57.129 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:16:57.379 INFO:teuthology.orchestra.run.smithi085.stdout:44 files removed 2023-01-25T05:16:57.390 INFO:teuthology.orchestra.run.smithi016.stdout:44 files removed 2023-01-25T05:16:57.426 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-radosgw 2023-01-25T05:16:57.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw 2023-01-25T05:16:57.758 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:16:57.789 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:16:58.417 INFO:teuthology.orchestra.run.smithi085.stdout:ceph packages for x86_64 263 kB/s | 80 kB 00:00 2023-01-25T05:16:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 289 kB/s | 80 kB 00:00 2023-01-25T05:16:58.717 INFO:teuthology.orchestra.run.smithi085.stdout:ceph noarch packages 35 kB/s | 9.3 kB 00:00 2023-01-25T05:16:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 39 kB/s | 9.3 kB 00:00 2023-01-25T05:16:58.974 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-01-25T05:16:59.026 INFO:teuthology.orchestra.run.smithi085.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-01-25T05:16:59.129 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T05:16:59.186 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-25T05:17:00.544 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 13 MB 00:01 2023-01-25T05:17:00.567 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 13 MB 00:01 2023-01-25T05:17:03.503 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 400 kB/s | 24 kB 00:00 2023-01-25T05:17:03.726 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 423 kB/s | 24 kB 00:00 2023-01-25T05:17:04.445 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-01-25T05:17:04.694 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-01-25T05:17:12.447 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-01-25T05:17:13.470 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-01-25T05:17:23.317 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:17:23.320 INFO:teuthology.orchestra.run.smithi085.stdout:==================================================================================================== 2023-01-25T05:17:23.320 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:17:23.320 INFO:teuthology.orchestra.run.smithi085.stdout:==================================================================================================== 2023-01-25T05:17:23.321 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:17:23.321 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T05:17:23.321 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrading: 2023-01-25T05:17:23.321 INFO:teuthology.orchestra.run.smithi085.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:17:23.321 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:17:23.322 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:17:23.322 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T05:17:23.322 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T05:17:23.322 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T05:17:23.322 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T05:17:23.323 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T05:17:23.323 INFO:teuthology.orchestra.run.smithi085.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T05:17:23.323 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T05:17:23.323 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T05:17:23.324 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T05:17:23.324 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T05:17:23.324 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T05:17:23.324 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T05:17:23.324 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T05:17:23.325 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T05:17:23.325 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T05:17:23.325 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T05:17:23.325 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T05:17:23.325 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T05:17:23.326 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T05:17:23.326 INFO:teuthology.orchestra.run.smithi085.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T05:17:23.326 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:17:23.326 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:17:23.326 INFO:teuthology.orchestra.run.smithi085.stdout:==================================================================================================== 2023-01-25T05:17:23.327 INFO:teuthology.orchestra.run.smithi085.stdout:Install 21 Packages 2023-01-25T05:17:23.327 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrade 2 Packages 2023-01-25T05:17:23.327 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:17:23.328 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 62 M 2023-01-25T05:17:23.328 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:17:24.456 INFO:teuthology.orchestra.run.smithi085.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 4.6 MB/s | 5.2 MB 00:01 2023-01-25T05:17:24.530 INFO:teuthology.orchestra.run.smithi085.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 329 kB/s | 24 kB 00:00 2023-01-25T05:17:24.785 INFO:teuthology.orchestra.run.smithi085.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 3.0 MB/s | 768 kB 00:00 2023-01-25T05:17:24.964 INFO:teuthology.orchestra.run.smithi085.stdout:(4/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 7.9 MB/s | 13 MB 00:01 2023-01-25T05:17:24.990 INFO:teuthology.orchestra.run.smithi085.stdout:(5/23): libradosstriper1-18.0.0-1959.gb0d25925. 2.3 MB/s | 474 kB 00:00 2023-01-25T05:17:25.000 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:17:25.002 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T05:17:25.003 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:17:25.003 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T05:17:25.003 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:17:25.004 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T05:17:25.004 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-01-25T05:17:25.005 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:17:25.005 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:17:25.006 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:17:25.006 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T05:17:25.006 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T05:17:25.007 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T05:17:25.007 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T05:17:25.008 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T05:17:25.008 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T05:17:25.008 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T05:17:25.009 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T05:17:25.009 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T05:17:25.010 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T05:17:25.010 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T05:17:25.011 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T05:17:25.011 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T05:17:25.011 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T05:17:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T05:17:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T05:17:25.013 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T05:17:25.013 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T05:17:25.013 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T05:17:25.014 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T05:17:25.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:17:25.014 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:17:25.015 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T05:17:25.015 INFO:teuthology.orchestra.run.smithi016.stdout:Install 21 Packages 2023-01-25T05:17:25.016 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2023-01-25T05:17:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:17:25.017 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 62 M 2023-01-25T05:17:25.017 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:17:25.056 INFO:teuthology.orchestra.run.smithi085.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 671 kB/s | 45 kB 00:00 2023-01-25T05:17:25.282 INFO:teuthology.orchestra.run.smithi085.stdout:(7/23): ceph-common-18.0.0-1959.gb0d25925.el8.x 12 MB/s | 24 MB 00:01 2023-01-25T05:17:25.307 INFO:teuthology.orchestra.run.smithi085.stdout:(8/23): python3-ceph-common-18.0.0-1959.gb0d259 458 kB/s | 115 kB 00:00 2023-01-25T05:17:25.799 INFO:teuthology.orchestra.run.smithi085.stdout:(9/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 7.8 MB/s | 6.5 MB 00:00 2023-01-25T05:17:25.825 INFO:teuthology.orchestra.run.smithi085.stdout:(10/23): python3-rados-18.0.0-1959.gb0d25925.el 720 kB/s | 372 kB 00:00 2023-01-25T05:17:25.883 INFO:teuthology.orchestra.run.smithi085.stdout:(11/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 4.3 MB/s | 364 kB 00:00 2023-01-25T05:17:25.917 INFO:teuthology.orchestra.run.smithi085.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 1.1 MB/s | 107 kB 00:00 2023-01-25T05:17:25.967 INFO:teuthology.orchestra.run.smithi085.stdout:(13/23): python3-cephfs-18.0.0-1959.gb0d25925.e 278 kB/s | 190 kB 00:00 2023-01-25T05:17:26.084 INFO:teuthology.orchestra.run.smithi085.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 352 kB/s | 59 kB 00:00 2023-01-25T05:17:26.101 INFO:teuthology.orchestra.run.smithi085.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-01-25T05:17:26.126 INFO:teuthology.orchestra.run.smithi085.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-01-25T05:17:26.178 INFO:teuthology.orchestra.run.smithi016.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 4.4 MB/s | 5.2 MB 00:01 2023-01-25T05:17:26.185 INFO:teuthology.orchestra.run.smithi085.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 561 kB/s | 47 kB 00:00 2023-01-25T05:17:26.210 INFO:teuthology.orchestra.run.smithi085.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-01-25T05:17:26.235 INFO:teuthology.orchestra.run.smithi085.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 878 kB/s | 44 kB 00:00 2023-01-25T05:17:26.269 INFO:teuthology.orchestra.run.smithi085.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-01-25T05:17:26.276 INFO:teuthology.orchestra.run.smithi016.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 248 kB/s | 24 kB 00:00 2023-01-25T05:17:26.361 INFO:teuthology.orchestra.run.smithi085.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-01-25T05:17:26.504 INFO:teuthology.orchestra.run.smithi016.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 3.3 MB/s | 768 kB 00:00 2023-01-25T05:17:26.671 INFO:teuthology.orchestra.run.smithi016.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 2.8 MB/s | 474 kB 00:00 2023-01-25T05:17:26.896 INFO:teuthology.orchestra.run.smithi016.stdout:(5/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 6.9 MB/s | 13 MB 00:01 2023-01-25T05:17:26.945 INFO:teuthology.orchestra.run.smithi085.stdout:(22/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 5.3 MB/s | 3.6 MB 00:00 2023-01-25T05:17:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 770 kB/s | 45 kB 00:00 2023-01-25T05:17:27.022 INFO:teuthology.orchestra.run.smithi016.stdout:(7/23): python3-ceph-common-18.0.0-1959.gb0d259 1.7 MB/s | 115 kB 00:00 2023-01-25T05:17:27.097 INFO:teuthology.orchestra.run.smithi016.stdout:(8/23): python3-cephfs-18.0.0-1959.gb0d25925.el 2.5 MB/s | 190 kB 00:00 2023-01-25T05:17:27.206 INFO:teuthology.orchestra.run.smithi016.stdout:(9/23): python3-rados-18.0.0-1959.gb0d25925.el8 3.4 MB/s | 372 kB 00:00 2023-01-25T05:17:27.281 INFO:teuthology.orchestra.run.smithi016.stdout:(10/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 4.8 MB/s | 364 kB 00:00 2023-01-25T05:17:27.369 INFO:teuthology.orchestra.run.smithi085.stdout:(23/23): librados2-18.0.0-1959.gb0d25925.el8.x8 3.2 MB/s | 3.6 MB 00:01 2023-01-25T05:17:27.372 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:17:27.372 INFO:teuthology.orchestra.run.smithi085.stdout:Total 15 MB/s | 62 MB 00:04 2023-01-25T05:17:27.417 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:17:27.462 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:17:27.463 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:17:27.556 INFO:teuthology.orchestra.run.smithi016.stdout:(11/23): ceph-common-18.0.0-1959.gb0d25925.el8. 9.6 MB/s | 24 MB 00:02 2023-01-25T05:17:27.581 INFO:teuthology.orchestra.run.smithi016.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 355 kB/s | 107 kB 00:00 2023-01-25T05:17:27.749 INFO:teuthology.orchestra.run.smithi016.stdout:(13/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_ 6.0 MB/s | 6.5 MB 00:01 2023-01-25T05:17:27.805 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:17:27.809 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:17:27.815 INFO:teuthology.orchestra.run.smithi016.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 251 kB/s | 59 kB 00:00 2023-01-25T05:17:27.866 INFO:teuthology.orchestra.run.smithi016.stdout:(15/23): gperftools-libs-2.7-9.el8.x86_64.rpm 991 kB/s | 306 kB 00:00 2023-01-25T05:17:27.924 INFO:teuthology.orchestra.run.smithi016.stdout:(16/23): libunwind-1.3.1-3.el8.x86_64.rpm 428 kB/s | 75 kB 00:00 2023-01-25T05:17:27.950 INFO:teuthology.orchestra.run.smithi016.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 562 kB/s | 47 kB 00:00 2023-01-25T05:17:28.000 INFO:teuthology.orchestra.run.smithi016.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-01-25T05:17:28.034 INFO:teuthology.orchestra.run.smithi016.stdout:(19/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2023-01-25T05:17:28.102 INFO:teuthology.orchestra.run.smithi016.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-01-25T05:17:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:(21/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-01-25T05:17:28.434 INFO:teuthology.orchestra.run.smithi016.stdout:(22/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 11 MB/s | 3.6 MB 00:00 2023-01-25T05:17:28.509 INFO:teuthology.orchestra.run.smithi016.stdout:(23/23): librados2-18.0.0-1959.gb0d25925.el8.x8 7.6 MB/s | 3.6 MB 00:00 2023-01-25T05:17:28.512 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:17:28.512 INFO:teuthology.orchestra.run.smithi016.stdout:Total 18 MB/s | 62 MB 00:03 2023-01-25T05:17:28.561 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:17:28.611 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:17:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:17:28.972 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:17:28.976 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:17:29.133 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:17:29.402 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T05:17:29.644 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:17:30.024 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:17:30.243 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:17:30.276 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:17:30.328 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:17:30.532 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T05:17:30.558 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:17:30.720 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:17:30.791 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T05:17:31.001 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:17:31.178 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:17:31.402 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:17:31.408 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:17:31.504 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:17:31.585 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:17:31.661 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:17:31.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:17:31.852 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:17:31.911 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:17:31.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T05:17:32.116 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T05:17:32.186 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:17:32.262 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T05:17:32.479 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:17:32.590 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:17:32.719 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:17:32.811 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:17:32.876 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:17:32.916 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:17:32.968 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:17:33.120 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T05:17:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:17:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:17:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T05:17:33.501 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T05:17:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T05:17:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:17:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:17:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:17:34.296 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T05:17:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:17:34.484 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:17:34.539 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T05:17:34.541 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:17:34.756 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T05:17:34.775 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T05:17:34.976 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T05:17:35.585 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T05:17:35.786 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:17:35.857 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:17:36.115 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T05:17:36.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T05:17:38.756 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:39.210 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:39.803 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:40.114 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:17:40.227 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:17:40.276 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:40.425 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:17:40.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:41.155 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:17:41.449 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:17:41.539 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:17:41.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:17:59.096 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:17:59.096 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /sys 2023-01-25T05:17:59.096 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /proc 2023-01-25T05:17:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2023-01-25T05:17:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /run 2023-01-25T05:17:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /mnt 2023-01-25T05:17:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /var/tmp 2023-01-25T05:17:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /home 2023-01-25T05:17:59.098 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /tmp 2023-01-25T05:17:59.098 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2023-01-25T05:17:59.098 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:00.477 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:18:00.498 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:18:00.498 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:18:00.724 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:18:00.725 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:01.079 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:18:01.080 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-01-25T05:18:01.080 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-01-25T05:18:01.080 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-01-25T05:18:01.081 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-01-25T05:18:01.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:01.189 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:01.189 INFO:teuthology.orchestra.run.smithi085.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T05:18:01.190 INFO:teuthology.orchestra.run.smithi085.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T05:18:01.190 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:01.342 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T05:18:01.342 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T05:18:01.343 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:18:01.343 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:18:01.343 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T05:18:01.343 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T05:18:01.344 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:18:01.344 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T05:18:01.344 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T05:18:01.344 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T05:18:01.345 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:18:01.345 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:18:01.345 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T05:18:01.345 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T05:18:01.345 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T05:18:01.346 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T05:18:01.346 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T05:18:01.346 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T05:18:01.346 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T05:18:01.346 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T05:18:01.347 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T05:18:01.347 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:18:01.347 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T05:18:01.347 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T05:18:02.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:18:02.567 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:18:02.568 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:18:02.773 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:02.773 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:51 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:18:02.774 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:03.232 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:03.232 INFO:teuthology.orchestra.run.smithi016.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T05:18:03.233 INFO:teuthology.orchestra.run.smithi016.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T05:18:03.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:03.394 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T05:18:03.394 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T05:18:03.394 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:18:03.394 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:18:03.395 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T05:18:03.395 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T05:18:03.395 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:18:03.395 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T05:18:03.396 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T05:18:03.396 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T05:18:03.396 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:18:03.396 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:18:03.396 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T05:18:03.397 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T05:18:03.397 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T05:18:03.397 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T05:18:03.397 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T05:18:03.397 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T05:18:03.398 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T05:18:03.398 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T05:18:03.398 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T05:18:03.398 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:18:03.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T05:18:03.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T05:18:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:18:04.894 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:52 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:05.209 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:18:05.403 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:05.404 INFO:teuthology.orchestra.run.smithi085.stdout:Upgraded: 2023-01-25T05:18:05.404 INFO:teuthology.orchestra.run.smithi085.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.404 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.404 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:18:05.405 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.405 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.405 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.405 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.405 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T05:18:05.406 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.406 INFO:teuthology.orchestra.run.smithi085.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T05:18:05.406 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T05:18:05.406 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.406 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T05:18:05.407 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.407 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T05:18:05.407 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T05:18:05.407 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.408 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.408 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.408 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T05:18:05.408 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.408 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.409 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:05.409 INFO:teuthology.orchestra.run.smithi085.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T05:18:05.409 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:05.409 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:18:05.616 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-test 2023-01-25T05:18:05.960 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:06.777 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:07.474 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:18:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-01-25T05:18:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:18:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T05:18:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T05:18:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T05:18:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T05:18:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T05:18:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T05:18:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T05:18:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T05:18:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:07.691 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:18:07.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-test 2023-01-25T05:18:08.182 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:18:08.183 INFO:teuthology.orchestra.run.smithi085.stdout:============================================================================================ 2023-01-25T05:18:08.183 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:18:08.184 INFO:teuthology.orchestra.run.smithi085.stdout:============================================================================================ 2023-01-25T05:18:08.184 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:18:08.184 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T05:18:08.184 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:18:08.185 INFO:teuthology.orchestra.run.smithi085.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T05:18:08.185 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T05:18:08.185 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T05:18:08.185 INFO:teuthology.orchestra.run.smithi085.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T05:18:08.185 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T05:18:08.186 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:08.186 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:18:08.186 INFO:teuthology.orchestra.run.smithi085.stdout:============================================================================================ 2023-01-25T05:18:08.186 INFO:teuthology.orchestra.run.smithi085.stdout:Install 6 Packages 2023-01-25T05:18:08.187 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:08.187 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 48 M 2023-01-25T05:18:08.188 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 218 M 2023-01-25T05:18:08.188 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:18:08.223 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:08.224 INFO:teuthology.orchestra.run.smithi085.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.8 MB/s | 70 kB 00:00 2023-01-25T05:18:08.300 INFO:teuthology.orchestra.run.smithi085.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-01-25T05:18:08.350 INFO:teuthology.orchestra.run.smithi085.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-01-25T05:18:08.400 INFO:teuthology.orchestra.run.smithi085.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-01-25T05:18:08.472 INFO:teuthology.orchestra.run.smithi085.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 479 kB/s | 137 kB 00:00 2023-01-25T05:18:09.047 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:10.587 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:18:10.588 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T05:18:10.589 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:18:10.589 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T05:18:10.589 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:18:10.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T05:18:10.589 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:18:10.590 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T05:18:10.590 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T05:18:10.590 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T05:18:10.590 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T05:18:10.590 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T05:18:10.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:10.591 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:18:10.591 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T05:18:10.591 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2023-01-25T05:18:10.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:10.592 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 48 M 2023-01-25T05:18:10.592 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 218 M 2023-01-25T05:18:10.593 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:18:10.626 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.9 MB/s | 70 kB 00:00 2023-01-25T05:18:10.675 INFO:teuthology.orchestra.run.smithi085.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 19 MB/s | 47 MB 00:02 2023-01-25T05:18:10.677 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:10.678 INFO:teuthology.orchestra.run.smithi085.stdout:Total 19 MB/s | 48 MB 00:02 2023-01-25T05:18:10.701 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-01-25T05:18:10.715 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:18:10.736 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:18:10.736 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:18:10.743 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.5 MB/s | 187 kB 00:00 2023-01-25T05:18:10.785 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-01-25T05:18:10.869 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 494 kB/s | 137 kB 00:00 2023-01-25T05:18:10.993 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:18:10.997 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:18:11.435 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:18:11.595 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:18:11.678 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:18:11.907 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T05:18:12.071 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T05:18:12.257 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T05:18:12.437 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:18:12.986 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 20 MB/s | 47 MB 00:02 2023-01-25T05:18:12.988 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:12.988 INFO:teuthology.orchestra.run.smithi016.stdout:Total 20 MB/s | 48 MB 00:02 2023-01-25T05:18:13.027 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:18:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:18:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:18:13.322 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:18:13.326 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:18:13.786 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:18:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:18:14.031 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:18:14.254 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T05:18:14.433 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T05:18:14.610 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T05:18:14.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:18:17.051 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:18:17.333 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:18:17.548 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:18:17.548 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T05:18:17.549 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T05:18:17.549 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T05:18:17.549 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T05:18:17.549 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T05:18:18.642 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T05:18:18.643 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:07 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:19.718 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:18:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:18:20.236 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:18:20.237 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T05:18:20.237 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T05:18:20.237 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T05:18:20.238 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T05:18:20.238 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T05:18:20.970 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:18:21.148 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:21.149 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:18:21.149 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:21.149 INFO:teuthology.orchestra.run.smithi085.stdout: jq-1.6-3.el8.x86_64 2023-01-25T05:18:21.150 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:21.150 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T05:18:21.150 INFO:teuthology.orchestra.run.smithi085.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T05:18:21.150 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T05:18:21.150 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:21.151 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:18:21.282 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph 2023-01-25T05:18:21.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T05:18:21.302 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:09 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:21.614 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:22.418 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:11 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:23.788 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:18:23.791 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================= 2023-01-25T05:18:23.791 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:18:23.792 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================= 2023-01-25T05:18:23.792 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:18:23.792 INFO:teuthology.orchestra.run.smithi085.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T05:18:23.792 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:18:23.793 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T05:18:23.793 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T05:18:23.793 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T05:18:23.794 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T05:18:23.794 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T05:18:23.794 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T05:18:23.794 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T05:18:23.795 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T05:18:23.795 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T05:18:23.795 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T05:18:23.795 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T05:18:23.795 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T05:18:23.796 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T05:18:23.796 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T05:18:23.796 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T05:18:23.796 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T05:18:23.796 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T05:18:23.797 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T05:18:23.797 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T05:18:23.797 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T05:18:23.797 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T05:18:23.797 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T05:18:23.798 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T05:18:23.798 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T05:18:23.798 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T05:18:23.798 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T05:18:23.798 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T05:18:23.799 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T05:18:23.799 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T05:18:23.799 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T05:18:23.799 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T05:18:23.799 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:23.800 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:18:23.800 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================= 2023-01-25T05:18:23.800 INFO:teuthology.orchestra.run.smithi085.stdout:Install 32 Packages 2023-01-25T05:18:23.800 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:23.801 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 28 M 2023-01-25T05:18:23.801 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 110 M 2023-01-25T05:18:23.802 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:18:23.928 INFO:teuthology.orchestra.run.smithi085.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2023-01-25T05:18:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:18:24.141 INFO:teuthology.orchestra.run.smithi085.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 4.5 MB/s | 1.5 MB 00:00 2023-01-25T05:18:24.195 INFO:teuthology.orchestra.run.smithi085.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 5.7 MB/s | 2.3 MB 00:00 2023-01-25T05:18:24.266 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:24.266 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:18:24.267 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:24.267 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-3.el8.x86_64 2023-01-25T05:18:24.267 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:24.267 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T05:18:24.267 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T05:18:24.268 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T05:18:24.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:24.268 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:18:24.287 INFO:teuthology.orchestra.run.smithi085.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.5 MB/s | 236 kB 00:00 2023-01-25T05:18:24.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph 2023-01-25T05:18:24.442 INFO:teuthology.orchestra.run.smithi085.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 282 kB/s | 44 kB 00:00 2023-01-25T05:18:24.563 INFO:teuthology.orchestra.run.smithi085.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.5 MB/s | 185 kB 00:00 2023-01-25T05:18:24.613 INFO:teuthology.orchestra.run.smithi085.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2023-01-25T05:18:24.696 INFO:teuthology.orchestra.run.smithi085.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 4.5 MB/s | 384 kB 00:00 2023-01-25T05:18:24.763 INFO:teuthology.orchestra.run.smithi085.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-01-25T05:18:24.766 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:24.805 INFO:teuthology.orchestra.run.smithi085.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 423 kB/s | 18 kB 00:00 2023-01-25T05:18:24.889 INFO:teuthology.orchestra.run.smithi085.stdout:(11/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 4.1 MB/s | 3.9 MB 00:00 2023-01-25T05:18:24.906 INFO:teuthology.orchestra.run.smithi085.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 486 kB/s | 49 kB 00:00 2023-01-25T05:18:25.090 INFO:teuthology.orchestra.run.smithi085.stdout:(13/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 18 MB/s | 17 MB 00:00 2023-01-25T05:18:25.115 INFO:teuthology.orchestra.run.smithi085.stdout:(14/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-01-25T05:18:25.165 INFO:teuthology.orchestra.run.smithi085.stdout:(15/32): python3-simplegeneric-0.8.1-17.el8.noa 375 kB/s | 19 kB 00:00 2023-01-25T05:18:25.199 INFO:teuthology.orchestra.run.smithi085.stdout:(16/32): python3-more-itertools-7.2.0-3.el8.noa 190 kB/s | 59 kB 00:00 2023-01-25T05:18:25.224 INFO:teuthology.orchestra.run.smithi085.stdout:(17/32): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2023-01-25T05:18:25.249 INFO:teuthology.orchestra.run.smithi085.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 286 kB/s | 24 kB 00:00 2023-01-25T05:18:25.274 INFO:teuthology.orchestra.run.smithi085.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 388 kB/s | 29 kB 00:00 2023-01-25T05:18:25.299 INFO:teuthology.orchestra.run.smithi085.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 365 kB/s | 27 kB 00:00 2023-01-25T05:18:25.384 INFO:teuthology.orchestra.run.smithi085.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.0 MB/s | 85 kB 00:00 2023-01-25T05:18:25.408 INFO:teuthology.orchestra.run.smithi085.stdout:(22/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.5 MB/s | 241 kB 00:00 2023-01-25T05:18:25.434 INFO:teuthology.orchestra.run.smithi085.stdout:(23/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2023-01-25T05:18:25.459 INFO:teuthology.orchestra.run.smithi085.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 307 kB/s | 23 kB 00:00 2023-01-25T05:18:25.509 INFO:teuthology.orchestra.run.smithi085.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 1.4 MB/s | 109 kB 00:00 2023-01-25T05:18:25.535 INFO:teuthology.orchestra.run.smithi085.stdout:(26/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-01-25T05:18:25.568 INFO:teuthology.orchestra.run.smithi085.stdout:(27/32): python3-cryptography-3.2.1-5.el8.x86_6 5.6 MB/s | 559 kB 00:00 2023-01-25T05:18:25.593 INFO:teuthology.orchestra.run.smithi085.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 664 kB/s | 39 kB 00:00 2023-01-25T05:18:25.594 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:25.627 INFO:teuthology.orchestra.run.smithi085.stdout:(29/32): python3-pytz-2017.2-9.el8.noarch.rpm 460 kB/s | 54 kB 00:00 2023-01-25T05:18:25.678 INFO:teuthology.orchestra.run.smithi085.stdout:(30/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-01-25T05:18:25.711 INFO:teuthology.orchestra.run.smithi085.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.2 MB/s | 103 kB 00:00 2023-01-25T05:18:25.736 INFO:teuthology.orchestra.run.smithi085.stdout:(32/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.1 MB/s | 457 kB 00:00 2023-01-25T05:18:25.737 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:25.738 INFO:teuthology.orchestra.run.smithi085.stdout:Total 15 MB/s | 28 MB 00:01 2023-01-25T05:18:25.820 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:18:25.857 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:18:25.857 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:18:26.061 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:18:26.064 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:18:27.161 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:18:27.164 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T05:18:27.165 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:18:27.165 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T05:18:27.165 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:18:27.165 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T05:18:27.166 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:18:27.166 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T05:18:27.166 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T05:18:27.166 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T05:18:27.166 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T05:18:27.167 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T05:18:27.167 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T05:18:27.167 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T05:18:27.167 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T05:18:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T05:18:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T05:18:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T05:18:27.168 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T05:18:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T05:18:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T05:18:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T05:18:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T05:18:27.169 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T05:18:27.170 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T05:18:27.170 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T05:18:27.170 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T05:18:27.170 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T05:18:27.170 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T05:18:27.171 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T05:18:27.171 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T05:18:27.171 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T05:18:27.171 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T05:18:27.171 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T05:18:27.172 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T05:18:27.172 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T05:18:27.172 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T05:18:27.172 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T05:18:27.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:27.173 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:18:27.173 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T05:18:27.173 INFO:teuthology.orchestra.run.smithi016.stdout:Install 32 Packages 2023-01-25T05:18:27.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:27.174 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 28 M 2023-01-25T05:18:27.174 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 110 M 2023-01-25T05:18:27.175 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:18:27.207 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:18:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 47 kB/s | 6.4 kB 00:00 2023-01-25T05:18:27.423 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T05:18:27.465 INFO:teuthology.orchestra.run.smithi016.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 5.2 MB/s | 1.5 MB 00:00 2023-01-25T05:18:27.590 INFO:teuthology.orchestra.run.smithi016.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 5.4 MB/s | 2.3 MB 00:00 2023-01-25T05:18:27.707 INFO:teuthology.orchestra.run.smithi016.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.0 MB/s | 236 kB 00:00 2023-01-25T05:18:27.742 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T05:18:27.849 INFO:teuthology.orchestra.run.smithi016.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 308 kB/s | 44 kB 00:00 2023-01-25T05:18:27.916 INFO:teuthology.orchestra.run.smithi016.stdout:(6/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86_ 6.4 MB/s | 3.9 MB 00:00 2023-01-25T05:18:27.966 INFO:teuthology.orchestra.run.smithi016.stdout:(7/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.5 MB/s | 185 kB 00:00 2023-01-25T05:18:27.979 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T05:18:28.059 INFO:teuthology.orchestra.run.smithi016.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 4.1 MB/s | 384 kB 00:00 2023-01-25T05:18:28.109 INFO:teuthology.orchestra.run.smithi016.stdout:(9/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 898 kB/s | 173 kB 00:00 2023-01-25T05:18:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:(10/32): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-01-25T05:18:28.160 INFO:teuthology.orchestra.run.smithi016.stdout:(11/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2023-01-25T05:18:28.185 INFO:teuthology.orchestra.run.smithi016.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 968 kB/s | 49 kB 00:00 2023-01-25T05:18:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:(13/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-01-25T05:18:28.227 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T05:18:28.260 INFO:teuthology.orchestra.run.smithi016.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-01-25T05:18:28.316 INFO:teuthology.orchestra.run.smithi016.stdout:(15/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-01-25T05:18:28.344 INFO:teuthology.orchestra.run.smithi016.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 223 kB/s | 19 kB 00:00 2023-01-25T05:18:28.370 INFO:teuthology.orchestra.run.smithi016.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 450 kB/s | 24 kB 00:00 2023-01-25T05:18:28.395 INFO:teuthology.orchestra.run.smithi016.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 584 kB/s | 29 kB 00:00 2023-01-25T05:18:28.420 INFO:teuthology.orchestra.run.smithi016.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 547 kB/s | 27 kB 00:00 2023-01-25T05:18:28.486 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T05:18:28.611 INFO:teuthology.orchestra.run.smithi016.stdout:(20/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 15 MB/s | 17 MB 00:01 2023-01-25T05:18:28.646 INFO:teuthology.orchestra.run.smithi016.stdout:(21/32): python3-waitress-1.2.1-2.el8.1.noarch. 961 kB/s | 241 kB 00:00 2023-01-25T05:18:28.671 INFO:teuthology.orchestra.run.smithi016.stdout:(22/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.0 MB/s | 251 kB 00:00 2023-01-25T05:18:28.687 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T05:18:28.696 INFO:teuthology.orchestra.run.smithi016.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 456 kB/s | 23 kB 00:00 2023-01-25T05:18:28.747 INFO:teuthology.orchestra.run.smithi016.stdout:(24/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-01-25T05:18:28.763 INFO:teuthology.orchestra.run.smithi016.stdout:(25/32): python3-webtest-2.0.33-1.el8.noarch.rp 562 kB/s | 85 kB 00:00 2023-01-25T05:18:28.789 INFO:teuthology.orchestra.run.smithi016.stdout:(26/32): python3-pycparser-2.14-14.el8.noarch.r 1.3 MB/s | 109 kB 00:00 2023-01-25T05:18:28.831 INFO:teuthology.orchestra.run.smithi016.stdout:(27/32): python3-cryptography-3.2.1-5.el8.x86_6 6.5 MB/s | 559 kB 00:00 2023-01-25T05:18:28.848 INFO:teuthology.orchestra.run.smithi016.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 776 kB/s | 39 kB 00:00 2023-01-25T05:18:28.865 INFO:teuthology.orchestra.run.smithi016.stdout:(29/32): python3-pytz-2017.2-9.el8.noarch.rpm 537 kB/s | 54 kB 00:00 2023-01-25T05:18:28.890 INFO:teuthology.orchestra.run.smithi016.stdout:(30/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T05:18:28.927 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T05:18:28.932 INFO:teuthology.orchestra.run.smithi016.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.5 MB/s | 103 kB 00:00 2023-01-25T05:18:28.957 INFO:teuthology.orchestra.run.smithi016.stdout:(32/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.1 MB/s | 457 kB 00:00 2023-01-25T05:18:28.959 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:28.959 INFO:teuthology.orchestra.run.smithi016.stdout:Total 16 MB/s | 28 MB 00:01 2023-01-25T05:18:29.053 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:18:29.093 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:18:29.094 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:18:29.311 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:18:29.315 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:18:29.910 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T05:18:30.245 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T05:18:30.450 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T05:18:30.459 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:18:30.641 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T05:18:30.680 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T05:18:30.867 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T05:18:31.036 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T05:18:31.125 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T05:18:31.241 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T05:18:31.324 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T05:18:31.480 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T05:18:31.502 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T05:18:31.653 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T05:18:31.756 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T05:18:31.812 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T05:18:31.943 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T05:18:31.973 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T05:18:32.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T05:18:32.203 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T05:18:32.421 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T05:18:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T05:18:32.631 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T05:18:32.874 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T05:18:32.911 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T05:18:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T05:18:33.199 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T05:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T05:18:33.395 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T05:18:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T05:18:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T05:18:33.750 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T05:18:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T05:18:34.055 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T05:18:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T05:18:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T05:18:34.363 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T05:18:34.440 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T05:18:34.505 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:18:34.576 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T05:18:34.845 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T05:18:35.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T05:18:35.234 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T05:18:35.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T05:18:35.820 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T05:18:35.996 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:18:36.213 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:18:36.596 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:18:36.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T05:18:36.784 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:18:36.996 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:18:37.047 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T05:18:37.192 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:18:37.209 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:18:37.314 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:18:37.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T05:18:37.678 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:18:37.678 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T05:18:37.678 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T05:18:37.678 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T05:18:37.679 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T05:18:37.679 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T05:18:37.679 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T05:18:37.679 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T05:18:37.679 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T05:18:37.680 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T05:18:37.680 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T05:18:37.680 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T05:18:37.680 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T05:18:37.681 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T05:18:37.681 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T05:18:37.681 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T05:18:37.681 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T05:18:37.681 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T05:18:37.682 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T05:18:37.682 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T05:18:37.682 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T05:18:37.682 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T05:18:37.682 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T05:18:37.683 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T05:18:37.683 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T05:18:37.683 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T05:18:37.684 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T05:18:37.684 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T05:18:37.684 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T05:18:37.685 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T05:18:37.685 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T05:18:37.685 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T05:18:37.772 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T05:18:37.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:18:39.167 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T05:18:39.167 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:28 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:39.535 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:18:39.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:18:40.149 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:18:40.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:18:40.529 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:18:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:18:40.729 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:18:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:18:41.190 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:18:41.190 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T05:18:41.191 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T05:18:41.191 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T05:18:41.191 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T05:18:41.191 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T05:18:41.191 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T05:18:41.192 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T05:18:41.192 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T05:18:41.192 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T05:18:41.192 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T05:18:41.192 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T05:18:41.193 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T05:18:41.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T05:18:41.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T05:18:41.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T05:18:41.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T05:18:41.194 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T05:18:41.195 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T05:18:41.195 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T05:18:41.195 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T05:18:41.195 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T05:18:41.196 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T05:18:41.196 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T05:18:41.196 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T05:18:41.196 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T05:18:41.197 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T05:18:41.732 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:18:41.917 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:41.917 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:18:41.918 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:41.918 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:41.918 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:41.918 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:18:41.919 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:41.919 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:41.919 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T05:18:41.919 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T05:18:41.919 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T05:18:41.920 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T05:18:41.920 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T05:18:41.920 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T05:18:41.920 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T05:18:41.921 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T05:18:41.921 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T05:18:41.921 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T05:18:41.921 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T05:18:41.921 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T05:18:41.922 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T05:18:41.922 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T05:18:41.922 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T05:18:41.922 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T05:18:41.922 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T05:18:41.923 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T05:18:41.923 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T05:18:41.923 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T05:18:41.923 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T05:18:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T05:18:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T05:18:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T05:18:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T05:18:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T05:18:41.925 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:41.925 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:18:42.076 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-base 2023-01-25T05:18:42.420 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:42.801 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T05:18:42.802 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:43.216 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:32 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:44.478 INFO:teuthology.orchestra.run.smithi085.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:18:44.588 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:18:44.589 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:18:44.589 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:18:44.709 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install cephadm 2023-01-25T05:18:45.050 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:45.290 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:18:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:45.494 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:18:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:18:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:18:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T05:18:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T05:18:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T05:18:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T05:18:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T05:18:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T05:18:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T05:18:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T05:18:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T05:18:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T05:18:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T05:18:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T05:18:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T05:18:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T05:18:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T05:18:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T05:18:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T05:18:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T05:18:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T05:18:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T05:18:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T05:18:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:45.501 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:18:45.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-base 2023-01-25T05:18:45.811 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:46.055 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:46.879 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:47.216 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:18:47.217 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:47.217 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:18:47.217 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:47.217 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:18:47.217 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T05:18:47.218 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:47.218 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:18:47.218 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:47.218 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:18:47.219 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:47.219 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 82 k 2023-01-25T05:18:47.219 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 374 k 2023-01-25T05:18:47.219 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:18:47.417 INFO:teuthology.orchestra.run.smithi085.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 412 kB/s | 82 kB 00:00 2023-01-25T05:18:47.418 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:47.418 INFO:teuthology.orchestra.run.smithi085.stdout:Total 410 kB/s | 82 kB 00:00 2023-01-25T05:18:47.419 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:18:47.426 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:18:47.426 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:18:47.428 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:18:47.431 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:18:47.478 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:18:48.346 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:18:48.467 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:18:48.468 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:18:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:18:48.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install cephadm 2023-01-25T05:18:48.972 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:49.114 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:49.298 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:49.465 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:49.789 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:50.547 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:50.548 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:51.294 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:18:51.295 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:18:51.296 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:18:51.296 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:18:51.296 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:18:51.296 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T05:18:51.296 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:51.297 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:18:51.297 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:18:51.298 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:18:51.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:51.298 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 82 k 2023-01-25T05:18:51.298 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 374 k 2023-01-25T05:18:51.298 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:18:51.473 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 466 kB/s | 82 kB 00:00 2023-01-25T05:18:51.474 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:51.474 INFO:teuthology.orchestra.run.smithi016.stdout:Total 463 kB/s | 82 kB 00:00 2023-01-25T05:18:51.475 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:18:51.483 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:18:51.483 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:18:51.485 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:18:51.489 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:18:51.530 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:18:52.938 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:18:53.114 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:53.115 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:18:53.115 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:18:53.116 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:53.116 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:18:53.261 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T05:18:53.618 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:18:54.432 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:43 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:54.634 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:54.793 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:54.964 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:55.804 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:18:55.805 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:55.805 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repo Size 2023-01-25T05:18:55.806 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:55.806 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:18:55.806 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T05:18:55.806 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:55.806 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:18:55.807 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:18:55.807 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:18:55.807 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:18:55.807 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 166 k 2023-01-25T05:18:55.807 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 503 k 2023-01-25T05:18:55.808 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:18:55.996 INFO:teuthology.orchestra.run.smithi085.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 870 kB/s | 166 kB 00:00 2023-01-25T05:18:55.997 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:18:55.997 INFO:teuthology.orchestra.run.smithi085.stdout:Total 865 kB/s | 166 kB 00:00 2023-01-25T05:18:55.998 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:18:56.008 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:18:56.009 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:18:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:18:56.046 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:18:56.061 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:18:56.064 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:18:56.193 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:18:56.339 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:18:56.694 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:18:57.686 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:18:57.687 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:46 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:18:58.418 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:18:58.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:18:58.612 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:18:58.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:18:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:18:58.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T05:18:59.187 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:00.061 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:48 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:00.088 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:00.267 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:00.267 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:00.268 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:19:00.268 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:00.268 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:00.399 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-mgr 2023-01-25T05:19:00.745 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:01.538 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:50 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:01.615 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:01.616 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:19:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T05:19:01.616 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:19:01.616 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:19:01.616 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T05:19:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:19:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:19:01.617 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:19:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:01.618 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 166 k 2023-01-25T05:19:01.618 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 503 k 2023-01-25T05:19:01.618 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:19:01.813 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 846 kB/s | 166 kB 00:00 2023-01-25T05:19:01.813 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:01.814 INFO:teuthology.orchestra.run.smithi016.stdout:Total 840 kB/s | 166 kB 00:00 2023-01-25T05:19:01.814 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:19:01.826 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:19:01.826 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:19:01.880 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:19:01.884 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:19:02.043 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:19:02.358 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:19:02.732 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:19:02.807 INFO:teuthology.orchestra.run.smithi085.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:19:02.915 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:02.916 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:19:02.916 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:03.025 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T05:19:03.370 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:03.782 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:19:03.782 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:51 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:04.148 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:53 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:05.538 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:05.539 INFO:teuthology.orchestra.run.smithi085.stdout:=================================================================================================== 2023-01-25T05:19:05.539 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:19:05.539 INFO:teuthology.orchestra.run.smithi085.stdout:=================================================================================================== 2023-01-25T05:19:05.539 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:19:05.539 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T05:19:05.540 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:19:05.540 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T05:19:05.540 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T05:19:05.540 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:19:05.540 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T05:19:05.541 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T05:19:05.541 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:05.541 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:19:05.541 INFO:teuthology.orchestra.run.smithi085.stdout:=================================================================================================== 2023-01-25T05:19:05.541 INFO:teuthology.orchestra.run.smithi085.stdout:Install 6 Packages 2023-01-25T05:19:05.542 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:05.542 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 2.0 M 2023-01-25T05:19:05.542 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 8.2 M 2023-01-25T05:19:05.543 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:19:05.692 INFO:teuthology.orchestra.run.smithi085.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 152 kB/s | 23 kB 00:00 2023-01-25T05:19:05.709 INFO:teuthology.orchestra.run.smithi085.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 87 kB/s | 15 kB 00:00 2023-01-25T05:19:05.860 INFO:teuthology.orchestra.run.smithi085.stdout:(3/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 5.4 MB/s | 1.7 MB 00:00 2023-01-25T05:19:05.877 INFO:teuthology.orchestra.run.smithi085.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 183 kB/s | 34 kB 00:00 2023-01-25T05:19:05.918 INFO:teuthology.orchestra.run.smithi085.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 728 kB/s | 43 kB 00:00 2023-01-25T05:19:05.952 INFO:teuthology.orchestra.run.smithi085.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 810 kB/s | 196 kB 00:00 2023-01-25T05:19:05.953 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:05.954 INFO:teuthology.orchestra.run.smithi085.stdout:Total 4.9 MB/s | 2.0 MB 00:00 2023-01-25T05:19:05.964 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:19:05.973 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:19:05.973 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:19:06.029 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:19:06.032 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:19:06.306 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:19:06.325 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:19:06.505 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T05:19:06.524 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:19:06.524 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:19:06.525 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:06.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr 2023-01-25T05:19:06.684 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T05:19:06.881 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T05:19:07.039 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:07.042 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T05:19:07.340 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T05:19:07.547 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:19:07.806 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:19:07.807 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T05:19:07.807 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T05:19:07.807 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T05:19:07.808 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T05:19:07.808 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T05:19:07.858 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:55 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:09.272 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:19:09.394 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:09.395 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:19:09.395 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:09.412 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T05:19:09.413 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:01:58 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:09.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T05:19:09.908 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:10.763 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:58 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:12.024 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:12.208 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:12.208 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:12.209 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:12.210 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:12.210 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:12.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T05:19:12.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T05:19:12.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T05:19:12.211 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:12.212 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:12.293 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:12.294 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T05:19:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:19:12.295 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T05:19:12.295 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:19:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T05:19:12.296 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:19:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T05:19:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T05:19:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:19:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2023-01-25T05:19:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:12.298 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 2.0 M 2023-01-25T05:19:12.298 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 8.2 M 2023-01-25T05:19:12.299 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:19:12.366 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T05:19:12.452 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 149 kB/s | 23 kB 00:00 2023-01-25T05:19:12.477 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 81 kB/s | 15 kB 00:00 2023-01-25T05:19:12.594 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 237 kB/s | 34 kB 00:00 2023-01-25T05:19:12.636 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 5.1 MB/s | 1.7 MB 00:00 2023-01-25T05:19:12.670 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-01-25T05:19:12.695 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 424 kB/s | 43 kB 00:00 2023-01-25T05:19:12.696 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:12.697 INFO:teuthology.orchestra.run.smithi016.stdout:Total 5.0 MB/s | 2.0 MB 00:00 2023-01-25T05:19:12.708 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:19:12.712 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:12.723 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:19:12.723 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:19:12.779 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:19:12.788 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:19:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:19:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T05:19:13.473 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T05:19:13.532 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:02 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:13.642 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T05:19:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T05:19:14.288 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T05:19:14.541 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:19:14.810 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:19:14.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T05:19:14.811 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T05:19:14.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T05:19:14.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T05:19:14.812 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T05:19:14.906 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:14.908 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================================= 2023-01-25T05:19:14.908 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:19:14.908 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================================= 2023-01-25T05:19:14.908 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:19:14.909 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T05:19:14.909 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:19:14.909 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T05:19:14.909 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T05:19:14.909 INFO:teuthology.orchestra.run.smithi085.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T05:19:14.910 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T05:19:14.910 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T05:19:14.910 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T05:19:14.910 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T05:19:14.910 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:14.911 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:19:14.911 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================================= 2023-01-25T05:19:14.911 INFO:teuthology.orchestra.run.smithi085.stdout:Install 8 Packages 2023-01-25T05:19:14.911 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:14.912 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 35 M 2023-01-25T05:19:14.912 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 210 M 2023-01-25T05:19:14.913 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:19:15.007 INFO:teuthology.orchestra.run.smithi085.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 6.5 MB/s | 644 kB 00:00 2023-01-25T05:19:15.050 INFO:teuthology.orchestra.run.smithi085.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2023-01-25T05:19:15.167 INFO:teuthology.orchestra.run.smithi085.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 32 MB/s | 3.7 MB 00:00 2023-01-25T05:19:15.218 INFO:teuthology.orchestra.run.smithi085.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 4.4 MB/s | 225 kB 00:00 2023-01-25T05:19:15.401 INFO:teuthology.orchestra.run.smithi085.stdout:(5/8): python3-scipy-1.0.0-21.module+el8.5.0+10 35 MB/s | 14 MB 00:00 2023-01-25T05:19:15.502 INFO:teuthology.orchestra.run.smithi085.stdout:(6/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.9 MB 00:00 2023-01-25T05:19:15.604 INFO:teuthology.orchestra.run.smithi085.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-01-25T05:19:15.869 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T05:19:15.870 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:03 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:17.344 INFO:teuthology.orchestra.run.smithi085.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 3.0 MB/s | 7.4 MB 00:02 2023-01-25T05:19:17.345 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:17.345 INFO:teuthology.orchestra.run.smithi085.stdout:Total 15 MB/s | 35 MB 00:02 2023-01-25T05:19:17.586 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:19:17.604 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:19:17.604 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:19:17.897 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:19:17.901 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:19:18.426 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:19:18.464 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:19:18.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:18.623 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:19:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T05:19:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T05:19:18.625 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T05:19:18.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:18.693 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:19:18.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T05:19:18.842 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:19:19.010 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:19.164 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:19.632 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:19.834 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:19:20.005 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:20.424 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:19:20.585 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:19:21.253 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:19:21.567 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T05:19:21.569 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:19:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T05:19:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:19:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T05:19:21.570 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:19:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T05:19:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T05:19:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T05:19:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T05:19:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T05:19:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T05:19:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T05:19:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:21.572 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:19:21.572 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T05:19:21.572 INFO:teuthology.orchestra.run.smithi016.stdout:Install 8 Packages 2023-01-25T05:19:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:21.573 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:19:21.573 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 35 M 2023-01-25T05:19:21.574 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 210 M 2023-01-25T05:19:21.574 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:19:21.652 INFO:teuthology.orchestra.run.smithi016.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 7.8 MB/s | 644 kB 00:00 2023-01-25T05:19:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2023-01-25T05:19:21.784 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:21.836 INFO:teuthology.orchestra.run.smithi016.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 25 MB/s | 3.7 MB 00:00 2023-01-25T05:19:21.878 INFO:teuthology.orchestra.run.smithi016.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T05:19:22.020 INFO:teuthology.orchestra.run.smithi016.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 34 MB/s | 4.9 MB 00:00 2023-01-25T05:19:22.195 INFO:teuthology.orchestra.run.smithi016.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 25 MB/s | 14 MB 00:00 2023-01-25T05:19:22.295 INFO:teuthology.orchestra.run.smithi016.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-01-25T05:19:23.543 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:24.537 INFO:teuthology.orchestra.run.smithi016.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 2.5 MB/s | 7.4 MB 00:02 2023-01-25T05:19:24.538 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:24.539 INFO:teuthology.orchestra.run.smithi016.stdout:Total 12 MB/s | 35 MB 00:02 2023-01-25T05:19:24.789 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:19:24.797 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T05:19:24.809 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:19:24.810 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:19:24.985 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:19:25.118 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:19:25.122 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:19:25.233 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:19:25.234 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T05:19:25.234 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:25.234 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T05:19:25.234 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T05:19:25.235 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:19:25.235 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:25.235 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T05:19:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:19:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:19:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:19:26.293 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T05:19:26.293 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:27.068 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:27.245 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:19:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:19:28.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:19:28.587 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:28.770 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:28.770 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:28.771 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:28.771 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:19:28.771 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:19:28.771 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T05:19:28.771 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T05:19:28.772 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T05:19:28.772 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T05:19:28.772 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T05:19:28.772 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:28.772 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:19:28.941 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-mgr-rook 2023-01-25T05:19:29.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:19:29.286 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:29.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:30.072 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:19 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:31.258 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:31.467 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:31.468 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================== 2023-01-25T05:19:31.469 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:19:31.469 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================== 2023-01-25T05:19:31.469 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:19:31.469 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T05:19:31.469 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:19:31.470 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T05:19:31.470 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T05:19:31.470 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T05:19:31.470 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T05:19:31.471 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T05:19:31.471 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T05:19:31.471 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T05:19:31.471 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T05:19:31.471 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T05:19:31.472 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:19:31.472 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T05:19:31.472 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T05:19:31.472 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:31.472 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:19:31.473 INFO:teuthology.orchestra.run.smithi085.stdout:======================================================================================================== 2023-01-25T05:19:31.473 INFO:teuthology.orchestra.run.smithi085.stdout:Install 13 Packages 2023-01-25T05:19:31.473 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:31.474 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 1.9 M 2023-01-25T05:19:31.474 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 25 M 2023-01-25T05:19:31.474 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:19:31.601 INFO:teuthology.orchestra.run.smithi085.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 126 kB/s | 16 kB 00:00 2023-01-25T05:19:31.626 INFO:teuthology.orchestra.run.smithi085.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 214 kB/s | 33 kB 00:00 2023-01-25T05:19:31.651 INFO:teuthology.orchestra.run.smithi085.stdout:(3/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 263 kB/s | 47 kB 00:00 2023-01-25T05:19:31.668 INFO:teuthology.orchestra.run.smithi085.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 1.2 MB/s | 81 kB 00:00 2023-01-25T05:19:31.719 INFO:teuthology.orchestra.run.smithi085.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T05:19:31.810 INFO:teuthology.orchestra.run.smithi085.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.6 MB/s | 155 kB 00:00 2023-01-25T05:19:31.836 INFO:teuthology.orchestra.run.smithi085.stdout:(7/13): python3-rsa-4.8-1.el8.noarch.rpm 334 kB/s | 61 kB 00:00 2023-01-25T05:19:31.861 INFO:teuthology.orchestra.run.smithi085.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 851 kB/s | 43 kB 00:00 2023-01-25T05:19:31.903 INFO:teuthology.orchestra.run.smithi085.stdout:(9/13): python3-kubernetes-11.0.0-6.el8.noarch. 4.2 MB/s | 1.2 MB 00:00 2023-01-25T05:19:31.928 INFO:teuthology.orchestra.run.smithi085.stdout:(10/13): python3-jsonpatch-1.21-2.el8.noarch.rp 432 kB/s | 27 kB 00:00 2023-01-25T05:19:31.953 INFO:teuthology.orchestra.run.smithi085.stdout:(11/13): python3-pyasn1-modules-0.3.7-6.el8.noa 938 kB/s | 110 kB 00:00 2023-01-25T05:19:31.979 INFO:teuthology.orchestra.run.smithi085.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 446 kB/s | 20 kB 00:00 2023-01-25T05:19:32.004 INFO:teuthology.orchestra.run.smithi085.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-01-25T05:19:32.004 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:32.005 INFO:teuthology.orchestra.run.smithi085.stdout:Total 3.6 MB/s | 1.9 MB 00:00 2023-01-25T05:19:32.063 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:19:32.078 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:19:32.078 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:19:32.148 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:19:32.152 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:19:32.571 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T05:19:32.636 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:19:32.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:19:32.849 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T05:19:33.020 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:19:33.021 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T05:19:33.021 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:19:33.021 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T05:19:33.021 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T05:19:33.022 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:19:33.022 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:19:33.022 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T05:19:33.101 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T05:19:33.264 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T05:19:33.440 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T05:19:33.625 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T05:19:33.853 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T05:19:34.085 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T05:19:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T05:19:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:22 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:34.298 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T05:19:34.446 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T05:19:34.670 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T05:19:35.776 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T05:19:36.016 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T05:19:36.199 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:19:36.429 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:19:36.429 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T05:19:36.429 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T05:19:36.430 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T05:19:36.430 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T05:19:36.430 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T05:19:36.430 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T05:19:36.430 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T05:19:36.431 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T05:19:36.431 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T05:19:36.431 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T05:19:36.431 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T05:19:36.431 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T05:19:36.612 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:19:36.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:36.816 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:19:36.816 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:36.816 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:19:36.816 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:19:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T05:19:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T05:19:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T05:19:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T05:19:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T05:19:36.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:36.818 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:36.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-rook 2023-01-25T05:19:37.341 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:37.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T05:19:37.510 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:26 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:38.166 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:26 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:39.752 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:39.754 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T05:19:39.754 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:19:39.755 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T05:19:39.755 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:19:39.755 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T05:19:39.756 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:19:39.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T05:19:39.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T05:19:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T05:19:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T05:19:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T05:19:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T05:19:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T05:19:39.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T05:19:39.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T05:19:39.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:19:39.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T05:19:39.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T05:19:39.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:39.760 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:19:39.760 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T05:19:39.760 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2023-01-25T05:19:39.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:39.761 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 1.9 M 2023-01-25T05:19:39.762 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 25 M 2023-01-25T05:19:39.762 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:19:39.868 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 300 kB/s | 33 kB 00:00 2023-01-25T05:19:39.885 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 128 kB/s | 16 kB 00:00 2023-01-25T05:19:39.927 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 280 kB/s | 47 kB 00:00 2023-01-25T05:19:39.952 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 969 kB/s | 81 kB 00:00 2023-01-25T05:19:40.002 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T05:19:40.082 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.9 MB/s | 155 kB 00:00 2023-01-25T05:19:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): python3-rsa-4.8-1.el8.noarch.rpm 334 kB/s | 61 kB 00:00 2023-01-25T05:19:40.137 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 789 kB/s | 43 kB 00:00 2023-01-25T05:19:40.187 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): python3-kubernetes-11.0.0-6.el8.noarch. 3.9 MB/s | 1.2 MB 00:00 2023-01-25T05:19:40.200 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:40.212 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): python3-jsonpatch-1.21-2.el8.noarch.rp 398 kB/s | 27 kB 00:00 2023-01-25T05:19:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): python3-pyasn1-modules-0.3.7-6.el8.noa 819 kB/s | 110 kB 00:00 2023-01-25T05:19:40.271 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 398 kB/s | 20 kB 00:00 2023-01-25T05:19:40.296 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-01-25T05:19:40.297 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:40.297 INFO:teuthology.orchestra.run.smithi016.stdout:Total 3.6 MB/s | 1.9 MB 00:00 2023-01-25T05:19:40.358 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:19:40.374 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:19:40.375 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:19:40.382 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:40.382 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:40.382 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:40.383 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T05:19:40.383 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T05:19:40.384 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T05:19:40.384 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T05:19:40.384 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T05:19:40.385 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T05:19:40.385 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T05:19:40.386 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T05:19:40.386 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T05:19:40.386 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T05:19:40.387 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T05:19:40.387 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T05:19:40.387 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:40.388 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:40.450 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:19:40.454 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:19:40.506 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T05:19:40.858 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:40.966 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:19:41.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T05:19:41.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T05:19:41.641 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:30 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:41.902 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T05:19:42.094 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T05:19:42.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T05:19:42.577 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T05:19:42.800 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T05:19:43.019 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:43.020 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================================== 2023-01-25T05:19:43.020 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:19:43.021 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================================== 2023-01-25T05:19:43.021 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:19:43.021 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T05:19:43.021 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:19:43.022 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T05:19:43.022 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T05:19:43.022 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T05:19:43.022 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T05:19:43.022 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:43.023 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:19:43.023 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================================== 2023-01-25T05:19:43.023 INFO:teuthology.orchestra.run.smithi085.stdout:Install 5 Packages 2023-01-25T05:19:43.023 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:43.024 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 6.0 M 2023-01-25T05:19:43.024 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 26 M 2023-01-25T05:19:43.024 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:19:43.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T05:19:43.178 INFO:teuthology.orchestra.run.smithi085.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 388 kB/s | 60 kB 00:00 2023-01-25T05:19:43.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T05:19:43.245 INFO:teuthology.orchestra.run.smithi085.stdout:(2/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 540 kB/s | 120 kB 00:00 2023-01-25T05:19:43.270 INFO:teuthology.orchestra.run.smithi085.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.9 MB/s | 476 kB 00:00 2023-01-25T05:19:43.304 INFO:teuthology.orchestra.run.smithi085.stdout:(4/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 4.2 MB/s | 538 kB 00:00 2023-01-25T05:19:43.395 INFO:teuthology.orchestra.run.smithi085.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 32 MB/s | 4.8 MB 00:00 2023-01-25T05:19:43.397 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:43.397 INFO:teuthology.orchestra.run.smithi085.stdout:Total 16 MB/s | 6.0 MB 00:00 2023-01-25T05:19:43.412 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T05:19:43.449 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:19:43.461 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:19:43.461 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:19:43.538 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:19:43.542 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:19:44.518 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:19:44.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T05:19:44.837 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T05:19:44.892 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T05:19:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:19:45.116 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T05:19:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:19:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T05:19:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T05:19:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T05:19:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T05:19:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T05:19:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T05:19:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T05:19:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T05:19:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T05:19:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T05:19:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T05:19:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T05:19:45.492 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:19:45.809 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T05:19:46.023 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:19:46.253 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:19:46.254 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T05:19:46.254 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T05:19:46.254 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:19:46.254 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T05:19:46.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T05:19:46.453 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:34 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:47.299 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T05:19:47.300 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:36 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:49.316 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:19:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:49.520 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:19:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T05:19:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T05:19:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T05:19:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T05:19:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T05:19:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T05:19:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T05:19:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T05:19:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T05:19:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T05:19:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T05:19:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T05:19:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:49.523 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:49.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T05:19:49.919 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:50.060 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:50.100 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:50.100 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:50.101 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:50.101 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T05:19:50.101 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T05:19:50.101 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T05:19:50.101 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T05:19:50.102 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:50.102 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:50.245 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-fuse 2023-01-25T05:19:50.598 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:50.893 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:51.423 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:40 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:52.470 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:19:52.471 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T05:19:52.472 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:19:52.473 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T05:19:52.473 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:19:52.473 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T05:19:52.473 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:19:52.473 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T05:19:52.474 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T05:19:52.474 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T05:19:52.474 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T05:19:52.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:52.474 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:19:52.475 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T05:19:52.475 INFO:teuthology.orchestra.run.smithi016.stdout:Install 5 Packages 2023-01-25T05:19:52.475 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:52.476 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 6.0 M 2023-01-25T05:19:52.476 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 26 M 2023-01-25T05:19:52.476 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:19:52.613 INFO:teuthology.orchestra.run.smithi016.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 434 kB/s | 60 kB 00:00 2023-01-25T05:19:52.663 INFO:teuthology.orchestra.run.smithi016.stdout:(2/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 635 kB/s | 120 kB 00:00 2023-01-25T05:19:52.697 INFO:teuthology.orchestra.run.smithi016.stdout:(3/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 6.3 MB/s | 538 kB 00:00 2023-01-25T05:19:52.730 INFO:teuthology.orchestra.run.smithi016.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2023-01-25T05:19:52.839 INFO:teuthology.orchestra.run.smithi016.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 27 MB/s | 4.8 MB 00:00 2023-01-25T05:19:52.839 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:52.840 INFO:teuthology.orchestra.run.smithi016.stdout:Total 16 MB/s | 6.0 MB 00:00 2023-01-25T05:19:52.871 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:19:52.872 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:19:52.872 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:19:52.872 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:19:52.872 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:19:52.873 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse 2023-01-25T05:19:52.873 INFO:teuthology.orchestra.run.smithi085.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T05:19:52.873 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2023-01-25T05:19:52.873 INFO:teuthology.orchestra.run.smithi085.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T05:19:52.873 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:52.874 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:19:52.874 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:19:52.874 INFO:teuthology.orchestra.run.smithi085.stdout:Install 2 Packages 2023-01-25T05:19:52.874 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:52.874 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 927 k 2023-01-25T05:19:52.875 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 2.9 M 2023-01-25T05:19:52.875 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:19:52.894 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:19:52.907 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:19:52.907 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:19:52.938 INFO:teuthology.orchestra.run.smithi085.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 1.3 MB/s | 83 kB 00:00 2023-01-25T05:19:52.989 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:19:52.993 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:19:53.154 INFO:teuthology.orchestra.run.smithi085.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 2.9 MB/s | 845 kB 00:00 2023-01-25T05:19:53.155 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:19:53.155 INFO:teuthology.orchestra.run.smithi085.stdout:Total 3.2 MB/s | 927 kB 00:00 2023-01-25T05:19:53.166 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:19:53.187 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:19:53.187 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:19:53.261 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:19:53.265 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:19:53.448 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:19:53.703 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T05:19:53.862 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:19:54.070 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:19:54.190 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:19:54.190 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T05:19:54.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T05:19:54.679 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T05:19:55.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:19:55.238 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T05:19:55.239 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:44 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T05:19:55.551 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:19:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:19:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T05:19:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T05:19:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:19:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T05:19:56.900 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T05:19:56.901 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:44 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:19:57.687 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:19:57.868 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:57.868 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:19:57.868 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T05:19:57.868 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:19:57.869 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:19:57.995 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-volume 2023-01-25T05:19:58.351 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:19:59.128 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:48 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:19:59.468 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:19:59.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:59.670 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:19:59.670 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:19:59.670 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T05:19:59.671 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T05:19:59.671 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T05:19:59.671 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T05:19:59.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:19:59.672 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:19:59.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-fuse 2023-01-25T05:20:00.173 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:00.587 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:00.587 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:00.587 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2023-01-25T05:20:00.588 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:00.588 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:00.588 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T05:20:00.588 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:00.588 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:00.589 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:00.589 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:00.589 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:00.590 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 263 k 2023-01-25T05:20:00.590 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 1.3 M 2023-01-25T05:20:00.590 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:00.811 INFO:teuthology.orchestra.run.smithi085.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-01-25T05:20:00.811 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:00.812 INFO:teuthology.orchestra.run.smithi085.stdout:Total 1.1 MB/s | 263 kB 00:00 2023-01-25T05:20:00.812 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:00.822 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:00.822 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:00.839 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:00.843 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:49 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:01.010 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:01.270 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:01.638 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:02.578 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:20:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:20:02.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse 2023-01-25T05:20:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T05:20:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T05:20:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T05:20:02.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:02.581 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:20:02.581 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:02.581 INFO:teuthology.orchestra.run.smithi016.stdout:Install 2 Packages 2023-01-25T05:20:02.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:02.582 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 927 k 2023-01-25T05:20:02.582 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 2.9 M 2023-01-25T05:20:02.582 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:20:02.655 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:02.655 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:51 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:02.974 INFO:teuthology.orchestra.run.smithi016.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 210 kB/s | 83 kB 00:00 2023-01-25T05:20:03.518 INFO:teuthology.orchestra.run.smithi016.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 902 kB/s | 845 kB 00:00 2023-01-25T05:20:03.519 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:03.519 INFO:teuthology.orchestra.run.smithi016.stdout:Total 988 kB/s | 927 kB 00:00 2023-01-25T05:20:03.533 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:20:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:20:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:20:03.640 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:20:03.644 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:20:04.208 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:20:04.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T05:20:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:20:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:20:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T05:20:05.204 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:20:05.383 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:05.384 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:20:05.384 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:20:05.384 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:05.384 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:05.515 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install librados-devel 2023-01-25T05:20:05.869 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:06.150 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T05:20:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:54 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:06.665 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:55 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:08.129 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:08.129 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:08.130 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repo Size 2023-01-25T05:20:08.130 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:08.130 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:08.130 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T05:20:08.130 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:08.131 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:08.131 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:08.131 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:08.131 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:08.132 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 139 k 2023-01-25T05:20:08.132 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 512 k 2023-01-25T05:20:08.132 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:08.335 INFO:teuthology.orchestra.run.smithi085.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 680 kB/s | 139 kB 00:00 2023-01-25T05:20:08.336 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:08.336 INFO:teuthology.orchestra.run.smithi085.stdout:Total 676 kB/s | 139 kB 00:00 2023-01-25T05:20:08.336 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:08.341 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:08.342 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:08.397 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:08.401 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:08.525 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:08.635 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:20:08.677 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:08.830 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:08.831 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:20:08.831 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T05:20:08.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:08.889 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:08.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-volume 2023-01-25T05:20:09.364 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:09.940 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:09.940 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:02:58 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:10.207 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:58 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:11.755 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:11.756 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:11.756 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T05:20:11.757 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:11.757 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:20:11.757 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T05:20:11.757 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:11.757 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:20:11.758 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:11.758 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:20:11.758 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:11.758 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 263 k 2023-01-25T05:20:11.758 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 1.3 M 2023-01-25T05:20:11.759 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:20:11.989 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 1.1 MB/s | 263 kB 00:00 2023-01-25T05:20:11.989 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:11.990 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.1 MB/s | 263 kB 00:00 2023-01-25T05:20:11.990 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:20:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:20:12.002 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:20:12.020 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:20:12.024 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:20:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:20:12.420 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:20:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:12.603 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:12.603 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:20:12.603 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:12.604 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:12.604 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:12.728 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install libcephfs2 2023-01-25T05:20:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:13.082 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:13.865 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:02 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:13.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:20:13.942 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:01 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:15.205 INFO:teuthology.orchestra.run.smithi085.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:15.311 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:15.312 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:15.312 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:15.422 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install libcephfs-devel 2023-01-25T05:20:15.773 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:16.492 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:20:16.634 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:05 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:16.691 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:20:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:20:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:16.692 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:16.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados-devel 2023-01-25T05:20:17.216 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:18.030 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:18.031 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:18.031 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repo Size 2023-01-25T05:20:18.031 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:18.031 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:18.032 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T05:20:18.032 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:18.032 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:18.032 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:18.032 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:18.033 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:18.033 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 30 k 2023-01-25T05:20:18.033 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 137 k 2023-01-25T05:20:18.033 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:18.042 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:06 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:18.420 INFO:teuthology.orchestra.run.smithi085.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 78 kB/s | 30 kB 00:00 2023-01-25T05:20:18.421 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:18.421 INFO:teuthology.orchestra.run.smithi085.stdout:Total 78 kB/s | 30 kB 00:00 2023-01-25T05:20:18.422 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:18.425 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:18.425 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:18.436 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:18.440 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:18.500 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:18.690 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:18.914 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:19.579 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:19.580 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:19.580 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T05:20:19.581 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:19.581 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:20:19.581 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T05:20:19.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:19.581 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:20:19.582 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:19.582 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:20:19.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:19.582 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 139 k 2023-01-25T05:20:19.582 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 512 k 2023-01-25T05:20:19.583 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:20:19.816 INFO:teuthology.orchestra.run.smithi016.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 592 kB/s | 139 kB 00:00 2023-01-25T05:20:19.817 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:19.818 INFO:teuthology.orchestra.run.smithi016.stdout:Total 588 kB/s | 139 kB 00:00 2023-01-25T05:20:19.818 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:20:19.824 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:20:19.824 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:20:19.880 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:19.880 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:08 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:19.881 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:20:19.885 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:20:20.014 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:20:20.187 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:20.435 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:21.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:22.208 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:20:22.386 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:22.387 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:20:22.387 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:22.387 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:22.387 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:22.511 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install librados2 2023-01-25T05:20:22.864 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:23.711 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:12 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:20:24.240 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:24.241 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:20:24.241 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:24.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:24.241 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:24.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs2 2023-01-25T05:20:24.772 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:25.055 INFO:teuthology.orchestra.run.smithi085.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:25.162 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:25.163 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:25.164 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:25.276 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install librbd1 2023-01-25T05:20:25.592 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:13 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:25.628 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:26.396 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:15 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:27.036 INFO:teuthology.orchestra.run.smithi016.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:27.153 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:27.154 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:27.154 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:27.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs-devel 2023-01-25T05:20:27.657 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:27.680 INFO:teuthology.orchestra.run.smithi085.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:27.788 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:27.789 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:27.789 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:27.917 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install python3-rados 2023-01-25T05:20:28.268 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:28.535 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:16 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:29.045 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:18 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:30.078 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:30.079 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:30.079 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T05:20:30.080 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:30.080 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:20:30.080 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T05:20:30.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:30.081 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:20:30.081 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:30.081 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:20:30.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:30.081 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 30 k 2023-01-25T05:20:30.082 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 137 k 2023-01-25T05:20:30.082 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:20:30.235 INFO:teuthology.orchestra.run.smithi016.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 197 kB/s | 30 kB 00:00 2023-01-25T05:20:30.236 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:30.236 INFO:teuthology.orchestra.run.smithi016.stdout:Total 195 kB/s | 30 kB 00:00 2023-01-25T05:20:30.236 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:20:30.240 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:20:30.240 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:20:30.251 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:20:30.255 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:20:30.312 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:20:30.339 INFO:teuthology.orchestra.run.smithi085.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:30.446 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:30.447 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:30.448 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:30.511 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:30.573 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install python3-rgw 2023-01-25T05:20:30.757 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:30.924 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:31.705 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:20 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:31.818 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:31.819 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:19 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:32.993 INFO:teuthology.orchestra.run.smithi085.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:33.101 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:33.102 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:33.103 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:33.216 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install python3-cephfs 2023-01-25T05:20:33.566 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:20:34.381 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:34.544 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:20:34.545 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:34.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:34.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados2 2023-01-25T05:20:35.079 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:35.643 INFO:teuthology.orchestra.run.smithi085.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:35.750 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:35.751 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:35.751 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:35.865 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install python3-rbd 2023-01-25T05:20:35.895 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:36.217 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:36.991 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:25 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:37.318 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:37.440 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:37.441 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:37.442 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librbd1 2023-01-25T05:20:37.959 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:38.274 INFO:teuthology.orchestra.run.smithi085.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:38.385 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:38.387 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2023-01-25T05:20:38.387 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:38.500 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install rbd-fuse 2023-01-25T05:20:38.772 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:26 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:38.851 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:39.640 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:28 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:40.211 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:40.329 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:40.330 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:40.331 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:40.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rados 2023-01-25T05:20:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:41.019 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:41.020 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:41.020 INFO:teuthology.orchestra.run.smithi085.stdout: Package Architecture Version Repository Size 2023-01-25T05:20:41.021 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:41.021 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:41.021 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:20:41.021 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:41.021 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:41.022 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:41.022 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:41.022 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:41.023 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 94 k 2023-01-25T05:20:41.023 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 280 k 2023-01-25T05:20:41.023 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:41.192 INFO:teuthology.orchestra.run.smithi085.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 554 kB/s | 94 kB 00:00 2023-01-25T05:20:41.193 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:41.193 INFO:teuthology.orchestra.run.smithi085.stdout:Total 550 kB/s | 94 kB 00:00 2023-01-25T05:20:41.193 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:41.203 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:41.203 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:41.257 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:41.261 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:41.409 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:41.561 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:41.672 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:29 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:41.812 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:42.841 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:42.841 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:31 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:43.107 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:43.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rgw 2023-01-25T05:20:43.740 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:44.589 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:32 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:45.196 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:20:45.377 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:45.377 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:20:45.377 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:45.377 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:45.378 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:45.501 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install rbd-mirror 2023-01-25T05:20:45.856 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:46.027 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:46.149 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:46.150 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:46.150 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:46.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-cephfs 2023-01-25T05:20:46.647 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:35 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:46.669 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:47.561 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:35 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:48.096 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:48.097 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:48.098 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repo Size 2023-01-25T05:20:48.098 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:48.098 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:48.098 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:20:48.099 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:48.099 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:48.099 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:48.099 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:48.099 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:48.100 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 3.4 M 2023-01-25T05:20:48.100 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 16 M 2023-01-25T05:20:48.100 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:48.461 INFO:teuthology.orchestra.run.smithi085.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 9.5 MB/s | 3.4 MB 00:00 2023-01-25T05:20:48.462 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:48.462 INFO:teuthology.orchestra.run.smithi085.stdout:Total 9.5 MB/s | 3.4 MB 00:00 2023-01-25T05:20:48.463 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:48.474 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:48.475 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:48.541 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:48.545 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:48.985 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:49.020 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:49.138 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:49.139 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:49.140 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:49.180 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:49.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rbd 2023-01-25T05:20:49.528 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:49.656 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:50.547 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:38 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:50.568 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:50.568 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:39 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:20:52.098 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:52.099 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T05:20:52.100 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:52.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-fuse 2023-01-25T05:20:52.610 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:52.948 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:20:53.127 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:53.127 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:20:53.127 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:53.127 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:53.128 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:20:53.271 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install rbd-nbd 2023-01-25T05:20:53.442 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:41 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:53.628 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2023-01-25T05:20:54.442 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:43 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:54.937 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:20:54.938 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:54.938 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-01-25T05:20:54.938 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:54.938 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:20:54.939 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:20:54.939 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:54.939 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:20:54.939 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:20:54.939 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:20:54.940 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:54.940 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 94 k 2023-01-25T05:20:54.940 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 280 k 2023-01-25T05:20:54.940 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:20:55.180 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 391 kB/s | 94 kB 00:00 2023-01-25T05:20:55.181 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:55.181 INFO:teuthology.orchestra.run.smithi016.stdout:Total 388 kB/s | 94 kB 00:00 2023-01-25T05:20:55.181 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:20:55.193 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:20:55.193 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:20:55.251 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:20:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:20:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:20:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:55.820 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2023-01-25T05:20:55.821 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:55.821 INFO:teuthology.orchestra.run.smithi085.stdout: Package Architecture Version Repository Size 2023-01-25T05:20:55.821 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:55.821 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2023-01-25T05:20:55.821 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:20:55.822 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:55.822 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2023-01-25T05:20:55.822 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2023-01-25T05:20:55.822 INFO:teuthology.orchestra.run.smithi085.stdout:Install 1 Package 2023-01-25T05:20:55.822 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:20:55.823 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 179 k 2023-01-25T05:20:55.823 INFO:teuthology.orchestra.run.smithi085.stdout:Installed size: 568 k 2023-01-25T05:20:55.823 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2023-01-25T05:20:56.081 INFO:teuthology.orchestra.run.smithi085.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 691 kB/s | 179 kB 00:00 2023-01-25T05:20:56.082 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:20:56.083 INFO:teuthology.orchestra.run.smithi085.stdout:Total 687 kB/s | 179 kB 00:00 2023-01-25T05:20:56.083 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2023-01-25T05:20:56.094 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2023-01-25T05:20:56.094 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2023-01-25T05:20:56.148 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2023-01-25T05:20:56.152 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2023-01-25T05:20:56.270 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2023-01-25T05:20:56.460 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:56.675 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:56.862 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:44 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:20:57.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:20:57.759 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:03:46 ago on Wed 25 Jan 2023 05:17:11 AM UTC. 2023-01-25T05:20:59.437 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:20:59.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:59.644 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:20:59.644 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:20:59.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:20:59.644 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:20:59.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-mirror 2023-01-25T05:21:00.095 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2023-01-25T05:21:00.168 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:21:00.272 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:00.273 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2023-01-25T05:21:00.273 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:21:00.273 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:00.273 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2023-01-25T05:21:00.412 DEBUG:teuthology.parallel:result is None 2023-01-25T05:21:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:48 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:21:02.554 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:21:02.555 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:02.555 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T05:21:02.556 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:02.556 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:21:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:21:02.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:02.557 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:21:02.557 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:02.557 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:21:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:02.557 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 3.4 M 2023-01-25T05:21:02.558 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 16 M 2023-01-25T05:21:02.558 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:21:02.934 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 9.1 MB/s | 3.4 MB 00:00 2023-01-25T05:21:02.934 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:21:02.935 INFO:teuthology.orchestra.run.smithi016.stdout:Total 9.1 MB/s | 3.4 MB 00:00 2023-01-25T05:21:02.935 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:21:02.948 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:21:02.949 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:21:03.019 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:21:03.023 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:21:03.489 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:21:03.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:04.046 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:05.284 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:53 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:21:07.997 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:21:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:08.191 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:21:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:21:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:08.192 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:21:08.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-nbd 2023-01-25T05:21:08.778 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T05:21:09.610 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:57 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:21:11.144 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T05:21:11.145 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:11.145 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-01-25T05:21:11.145 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:11.145 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T05:21:11.146 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:21:11.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:11.146 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T05:21:11.146 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T05:21:11.147 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T05:21:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:11.147 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 179 k 2023-01-25T05:21:11.147 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 568 k 2023-01-25T05:21:11.147 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T05:21:11.354 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 863 kB/s | 179 kB 00:00 2023-01-25T05:21:11.354 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:21:11.355 INFO:teuthology.orchestra.run.smithi016.stdout:Total 857 kB/s | 179 kB 00:00 2023-01-25T05:21:11.355 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T05:21:11.367 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T05:21:11.368 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T05:21:11.425 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T05:21:11.429 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T05:21:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T05:21:11.722 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:21:13.055 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:04:01 ago on Wed 25 Jan 2023 05:17:12 AM UTC. 2023-01-25T05:21:15.706 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T05:21:15.906 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:15.906 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T05:21:15.907 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:21:15.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:15.907 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T05:21:16.060 DEBUG:teuthology.parallel:result is None 2023-01-25T05:21:16.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T05:21:16.105 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T05:21:16.106 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:21:16.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:21:16.259 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:21:16.281 INFO:teuthology.orchestra.run.smithi016.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:21:16.282 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:21:16.282 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:21:16.283 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2023-01-25T05:21:16.332 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2023-01-25T05:21:16.334 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:21:16.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:21:16.477 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:21:16.501 INFO:teuthology.orchestra.run.smithi085.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:21:16.502 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:21:16.502 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:21:16.503 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T05:21:16.503 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:16.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:21:16.532 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:16.533 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:21:16.570 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T05:21:16.571 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:16.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:21:16.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:21:16.669 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:16.670 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:21:16.698 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:21:16.766 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T05:21:16.767 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:16.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:21:16.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:21:16.865 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:16.866 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:21:16.893 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:21:16.960 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T05:21:17.026 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T05:21:17.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-01-25T05:21:17.029 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 777 /var/log/ceph 2023-01-25T05:21:17.056 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T05:21:17.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:21:17.099 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:21:17.125 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T05:21:17.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:21:17.168 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:21:17.206 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T05:21:17.206 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T05:21:17.207 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136814', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136814', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi016.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi085.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 2']}}, {'rados': {'balance_reads': True, 'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T05:21:17.207 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:21:17.238 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:21:17.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:21:17.296 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:21:17.322 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:17.323 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:21:17.381 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:21:17.382 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-01-25T05:21:17.440 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:21:17.440 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 358553 Links: 1 2023-01-25T05:21:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 05:21:03.840508642 +0000 2023-01-25T05:21:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 05:16:27.786460765 +0000 2023-01-25T05:21:17.442 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 05:16:27.786460765 +0000 2023-01-25T05:21:17.442 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T05:21:17.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:21:17.508 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T05:21:17.508 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T05:21:17.508 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000124767 s, 4.1 MB/s 2023-01-25T05:21:17.509 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:21:17.569 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-01-25T05:21:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:21:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:17.629 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 350059 Links: 1 2023-01-25T05:21:17.629 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:17.629 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:17.629 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 05:21:03.840508642 +0000 2023-01-25T05:21:17.630 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 05:16:28.297453215 +0000 2023-01-25T05:21:17.630 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 05:16:28.297453215 +0000 2023-01-25T05:21:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T05:21:17.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:21:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T05:21:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T05:21:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00011623 s, 4.4 MB/s 2023-01-25T05:21:17.697 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:21:17.756 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-01-25T05:21:17.815 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:21:17.815 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:17.816 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 352244 Links: 1 2023-01-25T05:21:17.816 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:17.816 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:17.816 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 05:21:03.840508642 +0000 2023-01-25T05:21:17.816 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 05:16:28.801445768 +0000 2023-01-25T05:21:17.817 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 05:16:28.801445768 +0000 2023-01-25T05:21:17.817 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T05:21:17.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:21:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T05:21:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T05:21:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000117339 s, 4.4 MB/s 2023-01-25T05:21:17.884 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:21:17.943 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-01-25T05:21:18.002 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:21:18.003 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:18.003 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 350107 Links: 1 2023-01-25T05:21:18.003 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:18.003 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:18.004 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 05:21:03.840508642 +0000 2023-01-25T05:21:18.004 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 05:16:29.329437966 +0000 2023-01-25T05:21:18.004 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 05:16:29.329437966 +0000 2023-01-25T05:21:18.004 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T05:21:18.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:21:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T05:21:18.071 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T05:21:18.072 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000116117 s, 4.4 MB/s 2023-01-25T05:21:18.073 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:21:18.132 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:18.132 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:18.133 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:21:18.153 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:21:18.153 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_1 2023-01-25T05:21:18.211 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:21:18.212 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:18.212 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 351927 Links: 1 2023-01-25T05:21:18.212 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:18.212 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:18.213 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2023-01-25 05:20:49.337123108 +0000 2023-01-25T05:21:18.213 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2023-01-25 05:16:18.022371256 +0000 2023-01-25T05:21:18.213 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2023-01-25 05:16:18.022371256 +0000 2023-01-25T05:21:18.213 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2023-01-25T05:21:18.213 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:21:18.279 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2023-01-25T05:21:18.279 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2023-01-25T05:21:18.280 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000281243 s, 1.8 MB/s 2023-01-25T05:21:18.281 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:21:18.340 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_2 2023-01-25T05:21:18.397 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:21:18.398 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:18.398 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 352991 Links: 1 2023-01-25T05:21:18.398 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:18.398 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:18.399 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2023-01-25 05:20:49.337123108 +0000 2023-01-25T05:21:18.399 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2023-01-25 05:16:18.528365272 +0000 2023-01-25T05:21:18.399 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2023-01-25 05:16:18.528365272 +0000 2023-01-25T05:21:18.399 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2023-01-25T05:21:18.400 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:21:18.464 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2023-01-25T05:21:18.464 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2023-01-25T05:21:18.464 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000257477 s, 2.0 MB/s 2023-01-25T05:21:18.466 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:21:18.524 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_3 2023-01-25T05:21:18.583 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:21:18.583 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:18.583 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 359451 Links: 1 2023-01-25T05:21:18.583 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:18.584 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:18.584 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2023-01-25 05:20:49.337123108 +0000 2023-01-25T05:21:18.584 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2023-01-25 05:16:19.018359476 +0000 2023-01-25T05:21:18.584 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2023-01-25 05:16:19.018359476 +0000 2023-01-25T05:21:18.584 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2023-01-25T05:21:18.585 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:21:18.649 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2023-01-25T05:21:18.649 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2023-01-25T05:21:18.649 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000246811 s, 2.1 MB/s 2023-01-25T05:21:18.650 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:21:18.709 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_4 2023-01-25T05:21:18.766 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:21:18.767 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:21:18.767 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 358647 Links: 1 2023-01-25T05:21:18.767 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:21:18.768 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:21:18.768 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2023-01-25 05:20:49.337123108 +0000 2023-01-25T05:21:18.768 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2023-01-25 05:16:19.495353835 +0000 2023-01-25T05:21:18.768 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2023-01-25 05:16:19.495353835 +0000 2023-01-25T05:21:18.768 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2023-01-25T05:21:18.769 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:21:18.831 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2023-01-25T05:21:18.832 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2023-01-25T05:21:18.832 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000212246 s, 2.4 MB/s 2023-01-25T05:21:18.833 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:21:18.891 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:18.891 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:21:18.892 INFO:tasks.ceph:Generating config... 2023-01-25T05:21:18.893 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T05:21:18.893 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-25T05:21:18.894 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2023-01-25T05:21:18.894 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-25T05:21:18.894 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-25T05:21:18.894 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T05:21:18.895 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-25T05:21:18.895 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-25T05:21:18.895 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T05:21:18.895 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T05:21:18.896 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-25T05:21:18.896 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-25T05:21:18.897 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-25T05:21:18.897 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-25T05:21:18.897 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-25T05:21:18.897 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T05:21:18.898 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T05:21:18.898 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T05:21:18.898 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-25T05:21:18.898 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2023-01-25T05:21:18.899 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-01-25T05:21:18.899 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T05:21:18.899 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-01-25T05:21:18.900 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T05:21:18.900 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-25T05:21:18.900 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T05:21:18.900 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T05:21:18.901 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T05:21:18.901 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T05:21:18.901 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T05:21:18.901 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T05:21:18.902 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-25T05:21:18.902 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-25T05:21:18.902 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-25T05:21:18.902 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-01-25T05:21:18.903 INFO:tasks.ceph:[global] ms type = async 2023-01-25T05:21:18.903 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2023-01-25T05:21:18.903 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T05:21:18.903 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-01-25T05:21:18.904 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T05:21:18.904 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T05:21:18.904 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T05:21:18.904 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T05:21:18.905 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T05:21:18.905 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T05:21:18.905 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T05:21:18.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T05:21:18.964 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T05:21:18.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T05:21:19.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:21:19.046 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.c', '[v2:172.21.15.16:3301,v1:172.21.15.16:6790]'), ('mon.b', '172.21.15.85')] 2023-01-25T05:21:19.047 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],172.21.15.85', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T05:21:19.048 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:19.048 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:21:19.104 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --addv c '[v2:172.21.15.16:3301,v1:172.21.15.16:6790]' --add b 172.21.15.85 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:19.196 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e 2023-01-25T05:21:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-01-25T05:21:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-01-25T05:21:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:fsid ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e 2023-01-25T05:21:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-01-25T05:21:19.196917+0000 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-01-25T05:21:19.196917+0000 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] mon.b 2023-01-25T05:21:19.198 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] mon.c 2023-01-25T05:21:19.199 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T05:21:19.200 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:21:19.217 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e... 2023-01-25T05:21:19.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:21:19.260 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:21:19.304 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-01-25T05:21:19.305 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-01-25T05:21:19.305 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:21:19.305 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-01-25T05:21:19.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.306 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-01-25T05:21:19.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.306 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-01-25T05:21:19.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.306 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-01-25T05:21:19.307 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-01-25T05:21:19.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.307 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-01-25T05:21:19.307 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-01-25T05:21:19.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.308 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:21:19.308 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-01-25T05:21:19.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.308 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:21:19.308 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-01-25T05:21:19.309 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:21:19.310 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:21:19.310 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-01-25T05:21:19.310 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:21:19.310 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-01-25T05:21:19.310 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:21:19.311 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:21:19.311 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-01-25T05:21:19.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.311 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:21:19.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.312 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-01-25T05:21:19.312 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.312 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-01-25T05:21:19.312 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.312 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-01-25T05:21:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-01-25T05:21:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-01-25T05:21:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],172.21.15.85" 2023-01-25T05:21:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: mon client directed command retry = 5 2023-01-25T05:21:19.313 INFO:teuthology.orchestra.run.smithi016.stdout: mon election default strategy = 1 2023-01-25T05:21:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: ms bind msgr1 = False 2023-01-25T05:21:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: ms bind msgr2 = True 2023-01-25T05:21:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: ms inject socket failures = 5000 2023-01-25T05:21:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: ms type = async 2023-01-25T05:21:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-25T05:21:19.315 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_min_size = 2 2023-01-25T05:21:19.315 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_size = 2 2023-01-25T05:21:19.315 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e 2023-01-25T05:21:19.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.315 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-01-25T05:21:19.316 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-01-25T05:21:19.316 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.316 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-01-25T05:21:19.316 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-01-25T05:21:19.317 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-01-25T05:21:19.318 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-01-25T05:21:19.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.318 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-01-25T05:21:19.318 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-01-25T05:21:19.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-01-25T05:21:19.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:21:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-01-25T05:21:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-01-25T05:21:19.320 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-01-25T05:21:19.321 INFO:teuthology.orchestra.run.smithi016.stdout: debug monc = 20 2023-01-25T05:21:19.321 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-01-25T05:21:19.321 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore compression algorithm = lz4 2023-01-25T05:21:19.321 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore compression mode = aggressive 2023-01-25T05:21:19.321 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-01-25T05:21:19.322 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore zero block detection = True 2023-01-25T05:21:19.322 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-01-25T05:21:19.322 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-01-25T05:21:19.322 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T05:21:19.322 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: osd backoff on peering = True 2023-01-25T05:21:19.323 INFO:teuthology.orchestra.run.smithi016.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:21:19.324 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify cached snaps = True 2023-01-25T05:21:19.324 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify missing on start = True 2023-01-25T05:21:19.324 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-01-25T05:21:19.324 INFO:teuthology.orchestra.run.smithi016.stdout: osd heartbeat use min delay socket = True 2023-01-25T05:21:19.324 INFO:teuthology.orchestra.run.smithi016.stdout: osd inject bad map crc probability = 0.1 2023-01-25T05:21:19.325 INFO:teuthology.orchestra.run.smithi016.stdout: osd max markdown count = 1000 2023-01-25T05:21:19.325 INFO:teuthology.orchestra.run.smithi016.stdout: osd max scrubs = 3 2023-01-25T05:21:19.325 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock override recovery settings = True 2023-01-25T05:21:19.325 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-01-25T05:21:19.325 INFO:teuthology.orchestra.run.smithi016.stdout: osd op queue = debug_random 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: osd op queue cut off = debug_random 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: osd shutdown pgref assert = True 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.326 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-01-25T05:21:19.327 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T05:21:19.327 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-01-25T05:21:19.327 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-01-25T05:21:19.327 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-01-25T05:21:19.327 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:21:19.328 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:21:19.328 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-01-25T05:21:19.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.328 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-01-25T05:21:19.328 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-01-25T05:21:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:21:19.330 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-01-25T05:21:19.330 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-01-25T05:21:19.330 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:21:19.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.330 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:21:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:21:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:21:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:21:19.331 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:21:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:21:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-01-25T05:21:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:21:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout: mon scrub interval = 300 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-01-25T05:21:19.333 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-01-25T05:21:19.334 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-01-25T05:21:19.334 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:21:19.334 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:21:19.334 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-01-25T05:21:19.334 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-01-25T05:21:19.335 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-01-25T05:21:19.336 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2023-01-25T05:21:19.336 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2023-01-25T05:21:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:21:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2023-01-25T05:21:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2023-01-25T05:21:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2023-01-25T05:21:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2023-01-25T05:21:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2023-01-25T05:21:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2023-01-25T05:21:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2023-01-25T05:21:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:21:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2023-01-25T05:21:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:21:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:21:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:21:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2023-01-25T05:21:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:21:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2023-01-25T05:21:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:21:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:21:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: mon_allow_pool_size_one = true 2023-01-25T05:21:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:21:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2023-01-25T05:21:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2023-01-25T05:21:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2023-01-25T05:21:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2023-01-25T05:21:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],172.21.15.85" 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: mon client directed command retry = 5 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: mon election default strategy = 1 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: ms bind msgr1 = False 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: ms bind msgr2 = True 2023-01-25T05:21:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: ms inject socket failures = 5000 2023-01-25T05:21:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: ms type = async 2023-01-25T05:21:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-25T05:21:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_min_size = 2 2023-01-25T05:21:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_size = 2 2023-01-25T05:21:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e 2023-01-25T05:21:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.347 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2023-01-25T05:21:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2023-01-25T05:21:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2023-01-25T05:21:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2023-01-25T05:21:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2023-01-25T05:21:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2023-01-25T05:21:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2023-01-25T05:21:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2023-01-25T05:21:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:21:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2023-01-25T05:21:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2023-01-25T05:21:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:21:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2023-01-25T05:21:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2023-01-25T05:21:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2023-01-25T05:21:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: debug monc = 20 2023-01-25T05:21:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2023-01-25T05:21:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore compression algorithm = lz4 2023-01-25T05:21:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore compression mode = aggressive 2023-01-25T05:21:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2023-01-25T05:21:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore zero block detection = True 2023-01-25T05:21:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2023-01-25T05:21:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2023-01-25T05:21:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2023-01-25T05:21:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: osd backoff on peering = True 2023-01-25T05:21:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:21:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify cached snaps = True 2023-01-25T05:21:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify missing on start = True 2023-01-25T05:21:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2023-01-25T05:21:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: osd heartbeat use min delay socket = True 2023-01-25T05:21:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: osd inject bad map crc probability = 0.1 2023-01-25T05:21:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: osd max markdown count = 1000 2023-01-25T05:21:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: osd max scrubs = 3 2023-01-25T05:21:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: osd mclock override recovery settings = True 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd op queue = debug_random 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd op queue cut off = debug_random 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd shutdown pgref assert = True 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:21:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:21:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.358 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2023-01-25T05:21:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2023-01-25T05:21:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2023-01-25T05:21:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2023-01-25T05:21:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2023-01-25T05:21:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:21:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:21:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2023-01-25T05:21:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2023-01-25T05:21:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2023-01-25T05:21:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2023-01-25T05:21:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:21:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2023-01-25T05:21:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2023-01-25T05:21:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:21:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:21:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:21:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:21:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: # 1m isn't quite enough 2023-01-25T05:21:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:21:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:21:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon scrub interval = 300 2023-01-25T05:21:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: 2023-01-25T05:21:19.364 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2023-01-25T05:21:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2023-01-25T05:21:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2023-01-25T05:21:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2023-01-25T05:21:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:21:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:21:19.366 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2023-01-25T05:21:19.366 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2023-01-25T05:21:19.366 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2023-01-25T05:21:19.366 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T05:21:19.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T05:21:19.434 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T05:21:19.435 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:19.435 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T05:21:19.453 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:19.453 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T05:21:19.509 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:21:19.510 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:19.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:21:19.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:21:19.586 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:19.587 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:19.643 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:21:19.644 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:19.644 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:21:19.644 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:21:19.682 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:19.683 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:19.741 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T05:21:19.741 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T05:21:19.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:21:19.796 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:21:19.799 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:21:19.892 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:21:19.895 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T05:21:19.895 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T05:21:19.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T05:21:19.940 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T05:21:19.953 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T05:21:19.996 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T05:21:20.008 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T05:21:20.009 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:21:20.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T05:21:20.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:20.037 INFO:tasks.ceph:role: osd.0 2023-01-25T05:21:20.037 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:21:20.037 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:21:20.103 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:20.103 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:20.103 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:20.104 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:20.104 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:20.104 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:20.104 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:20.105 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:20.105 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:20.105 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:20.125 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T05:21:20.127 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T05:21:20.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T05:21:20.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T05:21:20.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T05:21:20.347 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:20.348 INFO:tasks.ceph:role: osd.1 2023-01-25T05:21:20.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:21:20.348 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:21:20.414 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:20.414 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:20.414 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:20.414 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:20.415 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:20.415 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:20.415 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:20.415 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:20.415 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:20.416 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:20.436 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T05:21:20.438 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T05:21:20.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T05:21:20.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T05:21:20.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T05:21:20.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:20.658 INFO:tasks.ceph:role: osd.2 2023-01-25T05:21:20.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:21:20.659 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:21:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:20.725 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:20.725 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:20.725 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:20.725 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:20.726 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:20.726 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:20.726 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:20.726 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:20.727 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:20.749 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T05:21:20.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T05:21:20.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T05:21:20.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T05:21:20.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T05:21:20.969 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:20.970 INFO:tasks.ceph:role: osd.3 2023-01-25T05:21:20.970 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:21:20.971 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:21:21.037 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:21.037 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:21.037 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:21.037 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:21.038 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:21.038 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:21.038 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:21.038 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:21.038 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:21.039 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:21.058 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T05:21:21.060 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T05:21:21.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T05:21:21.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T05:21:21.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.321+0000 7fabdce4a5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T05:21:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.322+0000 7fabdce4a5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T05:21:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.322+0000 7fabdce4a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:21:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.322+0000 7fabdce4a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:21:21.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.322+0000 7fabdce4a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:21:21.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.322+0000 7fabdce4a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T05:21:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:21.850+0000 7fabdce4a5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:22.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:22.612+0000 7fabdce4a5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:23.380+0000 7fabdce4a5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:24.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T05:21:24.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.284+0000 7fbda77a65c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T05:21:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.284+0000 7fbda77a65c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T05:21:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.284+0000 7fbda77a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:21:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.284+0000 7fbda77a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:21:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.284+0000 7fbda77a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:21:24.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.285+0000 7fbda77a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T05:21:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:24.814+0000 7fbda77a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:25.575+0000 7fbda77a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:26.340+0000 7fbda77a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:27.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T05:21:27.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.268+0000 7f6c88c3c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T05:21:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.269+0000 7f6c88c3c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T05:21:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.269+0000 7f6c88c3c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:21:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.269+0000 7f6c88c3c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:21:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.269+0000 7f6c88c3c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:21:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.269+0000 7f6c88c3c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T05:21:27.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:27.796+0000 7f6c88c3c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:28.559+0000 7f6c88c3c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:29.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:29.325+0000 7f6c88c3c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:30.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T05:21:30.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:30.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T05:21:30.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T05:21:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:21:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:21:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:21:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.249+0000 7f5a2e4565c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T05:21:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:30.776+0000 7f5a2e4565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:31.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:31.539+0000 7f5a2e4565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T05:21:32.304+0000 7f5a2e4565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:33.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T05:21:33.142 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T05:21:33.174 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:33.175 INFO:tasks.ceph:role: osd.4 2023-01-25T05:21:33.175 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:21:33.175 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:21:33.252 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:33.252 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:33.253 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:33.253 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:33.253 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:33.253 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:33.253 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:33.254 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:33.254 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:33.254 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:33.697 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2023-01-25T05:21:33.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2023-01-25T05:21:33.699 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T05:21:33.881 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T05:21:33.920 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T05:21:33.989 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:33.989 INFO:tasks.ceph:role: osd.5 2023-01-25T05:21:33.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:21:33.990 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:21:34.066 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:34.066 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:34.066 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:34.067 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:34.067 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:34.067 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:34.067 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:34.067 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:34.068 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:34.068 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:34.489 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2023-01-25T05:21:34.491 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2023-01-25T05:21:34.492 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T05:21:34.611 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T05:21:34.644 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T05:21:34.713 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:34.714 INFO:tasks.ceph:role: osd.6 2023-01-25T05:21:34.714 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:21:34.714 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:21:34.790 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:34.790 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:34.790 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:34.790 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:34.791 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:34.791 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:34.791 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:34.791 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:34.792 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:34.792 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:35.230 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2023-01-25T05:21:35.232 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2023-01-25T05:21:35.232 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T05:21:35.382 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T05:21:35.419 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T05:21:35.491 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:21:35.492 INFO:tasks.ceph:role: osd.7 2023-01-25T05:21:35.492 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com 2023-01-25T05:21:35.493 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:21:35.569 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:21:35.570 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:21:35.570 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:21:35.570 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:21:35.570 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:21:35.571 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:21:35.571 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:21:35.571 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:21:35.571 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:21:35.571 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:21:36.010 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2023-01-25T05:21:36.011 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2023-01-25T05:21:36.012 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T05:21:36.193 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T05:21:36.230 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:36.341 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.340+0000 7f3e9ee445c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T05:21:36.342 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.340+0000 7f3e9ee445c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T05:21:36.342 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.340+0000 7f3e9ee445c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:21:36.342 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.340+0000 7f3e9ee445c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:21:36.342 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.340+0000 7f3e9ee445c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:21:36.343 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.341+0000 7f3e9ee445c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T05:21:36.868 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:36.867+0000 7f3e9ee445c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:37.633 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:37.632+0000 7f3e9ee445c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:38.400 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:38.399+0000 7f3e9ee445c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:39.198 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T05:21:39.250 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:39.301 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T05:21:39.301 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T05:21:39.301 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:21:39.302 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:21:39.302 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:21:39.302 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.301+0000 7f3a698b75c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T05:21:39.831 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:39.831+0000 7f3a698b75c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:40.595 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:40.595+0000 7f3a698b75c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:41.359 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:41.360+0000 7f3a698b75c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:42.157 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T05:21:42.189 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:42.279 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.279+0000 7fc98e6295c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T05:21:42.280 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.279+0000 7fc98e6295c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T05:21:42.280 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.279+0000 7fc98e6295c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:21:42.280 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.279+0000 7fc98e6295c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:21:42.281 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.279+0000 7fc98e6295c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:21:42.281 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.280+0000 7fc98e6295c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T05:21:42.812 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:42.811+0000 7fc98e6295c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:43.579 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:43.578+0000 7fc98e6295c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:44.350 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:44.349+0000 7fc98e6295c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:45.149 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T05:21:45.180 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:45.270 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.269+0000 7f3ef31215c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T05:21:45.271 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.269+0000 7f3ef31215c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T05:21:45.271 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.270+0000 7f3ef31215c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:21:45.271 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.270+0000 7f3ef31215c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:21:45.271 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.270+0000 7f3ef31215c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:21:45.272 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.270+0000 7f3ef31215c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T05:21:45.798 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:45.798+0000 7f3ef31215c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:46.562 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:46.561+0000 7f3ef31215c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:47.328 INFO:teuthology.orchestra.run.smithi085.stderr:2023-01-25T05:21:47.327+0000 7f3ef31215c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:21:48.125 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T05:21:48.156 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T05:21:48.157 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T05:21:48.191 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T05:21:48.262 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T05:21:48.331 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T05:21:48.401 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T05:21:48.470 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.471 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T05:21:48.502 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.502 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T05:21:48.571 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.572 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T05:21:48.640 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.640 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T05:21:48.711 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.711 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T05:21:48.777 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:48.778 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T05:21:48.797 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:48.797 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T05:21:48.834 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T05:21:48.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:21:48.839 DEBUG:teuthology.orchestra.run.smithi085:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:21:48.877 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-01-25T05:21:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBPvNBjqhmFLxAAmTCdg7deg0a08GN93Cll9g== 2023-01-25T05:21:48.878 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-01-25T05:21:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBRvNBjtFNAExAASVJgnm4Zot/xJHQdu9auyg== 2023-01-25T05:21:48.878 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-01-25T05:21:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBUvNBj7HYIERAAbNX//1hZX3C3FRfPBtzu6A== 2023-01-25T05:21:48.879 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-01-25T05:21:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBXvNBjYCMJEBAACUqEGwpf2Ws0CDVFT1hEqA== 2023-01-25T05:21:48.879 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-01-25T05:21:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBavNBj06rrDhAA1Z+1KVDB+eER8aCiLav3bg== 2023-01-25T05:21:48.880 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-01-25T05:21:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBPvNBjFgU2NRAA53Heo8P/+EP27cZprVRSMg== 2023-01-25T05:21:48.880 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-01-25T05:21:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBgvNBjzv9XFBAAVTrB6EdmqpzcGICIotzwPQ== 2023-01-25T05:21:48.880 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-01-25T05:21:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBjvNBjY1j0ERAAVb5rCJfd1509OoF8vtHUEA== 2023-01-25T05:21:48.881 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-01-25T05:21:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBmvNBjDKyrEBAAARqJ8ZB3ONyhVZo0TF1t1A== 2023-01-25T05:21:48.881 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-01-25T05:21:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBpvNBjz78kEBAA3gL2Zsm8E3HKOup9SsHZBg== 2023-01-25T05:21:48.882 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-01-25T05:21:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBPvNBjSGQOOBAAr57C0SQzKG3o4vVOuWkZVQ== 2023-01-25T05:21:48.882 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-01-25T05:21:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBPvNBjEfVjOxAAcC34oKciRlHFdVW6qCIj6w== 2023-01-25T05:21:48.900 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.y] 2023-01-25T05:21:48.900 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBPvNBjqhmFLxAAmTCdg7deg0a08GN93Cll9g== 2023-01-25T05:21:48.900 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.0] 2023-01-25T05:21:48.901 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBRvNBjtFNAExAASVJgnm4Zot/xJHQdu9auyg== 2023-01-25T05:21:48.901 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.1] 2023-01-25T05:21:48.901 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBUvNBj7HYIERAAbNX//1hZX3C3FRfPBtzu6A== 2023-01-25T05:21:48.901 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.2] 2023-01-25T05:21:48.901 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBXvNBjYCMJEBAACUqEGwpf2Ws0CDVFT1hEqA== 2023-01-25T05:21:48.902 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.3] 2023-01-25T05:21:48.902 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBavNBj06rrDhAA1Z+1KVDB+eER8aCiLav3bg== 2023-01-25T05:21:48.902 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.x] 2023-01-25T05:21:48.902 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBPvNBjFgU2NRAA53Heo8P/+EP27cZprVRSMg== 2023-01-25T05:21:48.902 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.4] 2023-01-25T05:21:48.903 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBgvNBjzv9XFBAAVTrB6EdmqpzcGICIotzwPQ== 2023-01-25T05:21:48.903 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.5] 2023-01-25T05:21:48.903 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBjvNBjY1j0ERAAVb5rCJfd1509OoF8vtHUEA== 2023-01-25T05:21:48.903 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.6] 2023-01-25T05:21:48.903 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBmvNBjDKyrEBAAARqJ8ZB3ONyhVZo0TF1t1A== 2023-01-25T05:21:48.904 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.7] 2023-01-25T05:21:48.904 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBpvNBjz78kEBAA3gL2Zsm8E3HKOup9SsHZBg== 2023-01-25T05:21:48.904 INFO:teuthology.orchestra.run.smithi085.stdout:[client.0] 2023-01-25T05:21:48.904 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBPvNBjSGQOOBAAr57C0SQzKG3o4vVOuWkZVQ== 2023-01-25T05:21:48.904 INFO:teuthology.orchestra.run.smithi085.stdout:[client.1] 2023-01-25T05:21:48.905 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBPvNBjEfVjOxAAcC34oKciRlHFdVW6qCIj6w== 2023-01-25T05:21:48.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:21:48.926 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:21:48.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.001 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.057 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.112 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.168 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:21:49.223 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:21:49.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.278 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.334 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.388 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.444 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:21:49.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:21:49.500 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:21:49.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:21:49.558 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:21:49.614 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T05:21:49.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T05:21:49.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:21:49.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T05:21:49.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T05:21:49.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:21:49.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T05:21:49.962 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T05:21:49.990 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:21:50.089 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T05:21:50.115 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:50.118 DEBUG:teuthology.orchestra.run.smithi085:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:21:50.172 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T05:21:50.172 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T05:21:50.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T05:21:50.175 INFO:tasks.ceph.mon.a:Started 2023-01-25T05:21:50.176 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T05:21:50.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T05:21:50.178 INFO:tasks.ceph.mon.c:Started 2023-01-25T05:21:50.179 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T05:21:50.179 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T05:21:50.214 INFO:tasks.ceph.mon.b:Started 2023-01-25T05:21:50.214 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T05:21:50.215 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T05:21:50.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T05:21:50.219 INFO:tasks.ceph.mgr.y:Started 2023-01-25T05:21:50.219 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T05:21:50.220 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T05:21:50.222 INFO:tasks.ceph.mgr.x:Started 2023-01-25T05:21:50.222 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T05:21:50.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T05:21:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-01-25T05:21:55.667 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T05:21:55.668 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:55.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T05:21:55.671 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:55.671+0000 7f9579bf1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:21:55.692 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:55.691+0000 7f5cb9281200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:21:55.696 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:55.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T05:21:55.735 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:55.734+0000 7f9579bf1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:21:55.761 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:55.760+0000 7f5cb9281200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:21:55.763 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:55.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T05:21:55.809 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:55.808+0000 7f9579bf1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:21:55.831 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T05:21:55.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T05:21:55.841 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:55.840+0000 7f5cb9281200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:21:55.868 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:55.869 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T05:21:55.886 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:55.886+0000 7f9579bf1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:21:55.895 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:55.895 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T05:21:55.925 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:55.925+0000 7f5cb9281200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:21:55.962 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:55.962 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T05:21:56.030 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2023-01-25T05:21:56.030 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T05:21:56.038 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:56.037+0000 7f9579bf1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:21:56.063 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 88c5b9c1-6971-40ba-a114-73b6f37cb2fa 0 2023-01-25T05:21:56.071 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:56.071+0000 7f5cb9281200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:21:56.319 INFO:teuthology.orchestra.run.smithi085.stdout:0 2023-01-25T05:21:56.332 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new c6febc44-a227-44be-913a-e47a15762932 1 2023-01-25T05:21:56.390 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:56.389+0000 7f9579bf1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:21:56.432 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:56.431+0000 7f5cb9281200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:21:56.469 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:56.468+0000 7f9579bf1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:21:56.509 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:56.508+0000 7f5cb9281200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:21:56.549 INFO:teuthology.orchestra.run.smithi085.stdout:1 2023-01-25T05:21:56.562 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 04ce138a-7d40-4c56-87e2-dd95b27be578 2 2023-01-25T05:21:56.601 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:56.600+0000 7f9579bf1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:21:56.642 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:56.641+0000 7f5cb9281200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:21:56.685 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:56.684+0000 7f9579bf1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:21:56.738 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:56.737+0000 7f5cb9281200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:21:56.780 INFO:teuthology.orchestra.run.smithi085.stdout:2 2023-01-25T05:21:56.792 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new e1bfe8de-ea8b-4212-9de2-af8b9b83794e 3 2023-01-25T05:21:57.045 INFO:teuthology.orchestra.run.smithi085.stdout:3 2023-01-25T05:21:57.057 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new d8d054e5-59ef-447b-97bc-0c81e47da65c 4 2023-01-25T05:21:57.161 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:57.161+0000 7f9579bf1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:21:57.196 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:57.196+0000 7f5cb9281200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:21:57.273 INFO:teuthology.orchestra.run.smithi085.stdout:4 2023-01-25T05:21:57.286 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 02367b52-2c78-4576-b20f-71e603f35ce6 5 2023-01-25T05:21:57.511 INFO:teuthology.orchestra.run.smithi085.stdout:5 2023-01-25T05:21:57.524 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new cd5c9527-4467-4828-9583-9851d083d931 6 2023-01-25T05:21:57.754 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:57.753+0000 7f9579bf1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:21:57.762 INFO:teuthology.orchestra.run.smithi085.stdout:6 2023-01-25T05:21:57.775 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 918ee93e-d6c7-42c0-8112-f7cacc891ce0 7 2023-01-25T05:21:57.798 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:57.797+0000 7f5cb9281200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:21:57.916 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:57.915+0000 7f9579bf1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:21:57.958 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:57.957+0000 7f5cb9281200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:21:57.991 INFO:teuthology.orchestra.run.smithi085.stdout:7 2023-01-25T05:21:58.004 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T05:21:58.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T05:21:58.007 INFO:tasks.ceph.osd.0:Started 2023-01-25T05:21:58.007 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T05:21:58.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T05:21:58.010 INFO:tasks.ceph.osd.1:Started 2023-01-25T05:21:58.010 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T05:21:58.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T05:21:58.013 INFO:tasks.ceph.osd.2:Started 2023-01-25T05:21:58.013 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T05:21:58.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T05:21:58.016 INFO:tasks.ceph.osd.3:Started 2023-01-25T05:21:58.016 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T05:21:58.016 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T05:21:58.019 INFO:tasks.ceph.osd.4:Started 2023-01-25T05:21:58.019 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T05:21:58.019 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T05:21:58.021 INFO:tasks.ceph.osd.5:Started 2023-01-25T05:21:58.022 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T05:21:58.022 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T05:21:58.024 INFO:tasks.ceph.osd.6:Started 2023-01-25T05:21:58.024 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T05:21:58.025 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T05:21:58.027 INFO:tasks.ceph.osd.7:Started 2023-01-25T05:21:58.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:21:58.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:21:58.265 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e","created":"2023-01-25T05:21:55.524643+0000","modified":"2023-01-25T05:21:57.989718+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:21:57.989718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"88c5b9c1-6971-40ba-a114-73b6f37cb2fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c6febc44-a227-44be-913a-e47a15762932","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"04ce138a-7d40-4c56-87e2-dd95b27be578","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e1bfe8de-ea8b-4212-9de2-af8b9b83794e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8d054e5-59ef-447b-97bc-0c81e47da65c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"02367b52-2c78-4576-b20f-71e603f35ce6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cd5c9527-4467-4828-9583-9851d083d931","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"918ee93e-d6c7-42c0-8112-f7cacc891ce0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:21:58.277 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T05:21:58.278 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T05:21:58.620 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:21:58.619+0000 7fbbe26495c0 -1 Falling back to public interface 2023-01-25T05:21:58.625 INFO:tasks.ceph.osd.7.smithi085.stderr:2023-01-25T05:21:58.624+0000 7f26e1c335c0 -1 Falling back to public interface 2023-01-25T05:21:58.628 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:21:58.627+0000 7f74b53fb5c0 -1 Falling back to public interface 2023-01-25T05:21:58.634 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:58.633+0000 7f9579bf1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:21:58.635 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:21:58.634+0000 7f24174595c0 -1 Falling back to public interface 2023-01-25T05:21:58.636 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:21:58.635+0000 7f243d93b5c0 -1 Falling back to public interface 2023-01-25T05:21:58.644 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:21:58.643+0000 7f9ef353b5c0 -1 Falling back to public interface 2023-01-25T05:21:58.658 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:21:58.657+0000 7f15d19765c0 -1 Falling back to public interface 2023-01-25T05:21:58.660 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:21:58.659+0000 7fe83e8af5c0 -1 Falling back to public interface 2023-01-25T05:21:58.710 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:58.709+0000 7f5cb9281200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:21:58.712 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:58.711+0000 7f9579bf1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:21:58.789 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:58.788+0000 7f5cb9281200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:21:58.850 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:58.849+0000 7f9579bf1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:21:58.931 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:58.930+0000 7f5cb9281200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:21:59.010 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.009+0000 7f9579bf1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:21:59.083 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:21:59.095 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:59.095+0000 7f5cb9281200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:21:59.138 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.137+0000 7f9579bf1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:21:59.237 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:59.236+0000 7f5cb9281200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:21:59.278 INFO:teuthology.misc.health.smithi016.stdout: 2023-01-25T05:21:59.279 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e","created":"2023-01-25T05:21:55.524643+0000","modified":"2023-01-25T05:21:57.989718+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:21:57.989718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"88c5b9c1-6971-40ba-a114-73b6f37cb2fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c6febc44-a227-44be-913a-e47a15762932","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"04ce138a-7d40-4c56-87e2-dd95b27be578","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e1bfe8de-ea8b-4212-9de2-af8b9b83794e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8d054e5-59ef-447b-97bc-0c81e47da65c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"02367b52-2c78-4576-b20f-71e603f35ce6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cd5c9527-4467-4828-9583-9851d083d931","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"918ee93e-d6c7-42c0-8112-f7cacc891ce0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:21:59.289 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T05:21:59.489 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.488+0000 7f9579bf1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:21:59.602 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:59.601+0000 7f5cb9281200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:21:59.641 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.640+0000 7f9579bf1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:21:59.755 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.754+0000 7f9579bf1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:21:59.764 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:59.761+0000 7f5cb9281200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:21:59.881 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:21:59.881+0000 7f5cb9281200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:21:59.902 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:21:59.902+0000 7f9579bf1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:22:00.002 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:22:00.001+0000 7f9579bf1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:22:00.034 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:22:00.033+0000 7f5cb9281200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:22:00.137 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:22:00.136+0000 7f5cb9281200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:22:00.378 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:22:00.377+0000 7f9579bf1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:22:00.505 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:22:00.504+0000 7f5cb9281200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:22:00.958 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:22:00.957+0000 7f9579bf1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:22:01.123 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-01-25T05:22:01.122+0000 7f5cb9281200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:22:01.982 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:22:01.981+0000 7f9531f72700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T05:22:01.983 INFO:tasks.ceph.mgr.x.smithi085.stderr:2023-01-25T05:22:01.982+0000 7f9531f72700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T05:22:02.112 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:02.111+0000 7f74b53fb5c0 -1 osd.0 0 log_to_monitors true 2023-01-25T05:22:02.133 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:22:02.132+0000 7fbbe26495c0 -1 osd.4 0 log_to_monitors true 2023-01-25T05:22:02.139 INFO:tasks.ceph.osd.7.smithi085.stderr:2023-01-25T05:22:02.139+0000 7f26e1c335c0 -1 osd.7 0 log_to_monitors true 2023-01-25T05:22:02.146 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:22:02.145+0000 7f24174595c0 -1 osd.6 0 log_to_monitors true 2023-01-25T05:22:02.154 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:22:02.153+0000 7f9ef353b5c0 -1 osd.1 0 log_to_monitors true 2023-01-25T05:22:02.164 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:22:02.163+0000 7f243d93b5c0 -1 osd.2 0 log_to_monitors true 2023-01-25T05:22:02.165 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:22:02.164+0000 7f15d19765c0 -1 osd.3 0 log_to_monitors true 2023-01-25T05:22:02.180 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:02.180+0000 7fe83e8af5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T05:22:03.991 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:22:03.990+0000 7f9ee78f3700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T05:22:03.991 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:22:03.990+0000 7f15c5d2e700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T05:22:03.992 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:03.990+0000 7f74a97b3700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T05:22:03.992 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:22:03.991+0000 7f9edb61f700 -1 osd.1 0 handle_osd_map injecting map crc failure 2023-01-25T05:22:03.994 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:22:03.993+0000 7f240b811700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T05:22:03.994 INFO:tasks.ceph.osd.7.smithi085.stderr:2023-01-25T05:22:03.993+0000 7f26d5feb700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T05:22:03.995 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:03.993+0000 7fe832c67700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T05:22:03.995 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:22:03.993+0000 7fbbd6a01700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T05:22:03.995 INFO:tasks.ceph.osd.7.smithi085.stderr:2023-01-25T05:22:03.994+0000 7f26c9d17700 -1 osd.7 0 handle_osd_map injecting map crc failure 2023-01-25T05:22:03.996 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:22:03.994+0000 7f23ff53d700 -1 osd.6 0 handle_osd_map injecting map crc failure 2023-01-25T05:22:03.996 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:22:03.994+0000 7fbbca72d700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-01-25T05:22:03.997 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:22:03.993+0000 7f2431cf3700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T05:22:03.997 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:22:03.995+0000 7f2425a1f700 -1 osd.2 0 handle_osd_map injecting map crc failure 2023-01-25T05:22:03.997 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:22:03.995+0000 7f9edfcce700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:03.998 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:22:03.997+0000 7f15be109700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:03.999 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:03.999+0000 7f74a1b8e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:04.001 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:22:04.000+0000 7f242a0ce700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:04.002 INFO:tasks.ceph.osd.7.smithi085.stderr:2023-01-25T05:22:04.001+0000 7f26ce3c6700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:04.002 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:04.001+0000 7fe82b042700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:04.003 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:22:04.002+0000 7fbbceddc700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:04.003 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:22:04.002+0000 7f2403bec700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:22:06.096 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:22:06.429 INFO:teuthology.misc.health.smithi016.stdout: 2023-01-25T05:22:06.429 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":14,"fsid":"ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e","created":"2023-01-25T05:21:55.524643+0000","modified":"2023-01-25T05:22:06.002083+0000","last_up_change":"2023-01-25T05:22:04.995341+0000","last_in_change":"2023-01-25T05:21:57.989718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:22:05.040275+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"88c5b9c1-6971-40ba-a114-73b6f37cb2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":92923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6801","nonce":92923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6803","nonce":92923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":92923}]},"public_addr":"172.21.15.16:6800/92923","cluster_addr":"172.21.15.16:6801/92923","heartbeat_back_addr":"172.21.15.16:6803/92923","heartbeat_front_addr":"172.21.15.16:6802/92923","state":["exists","up"]},{"osd":1,"uuid":"c6febc44-a227-44be-913a-e47a15762932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":92924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":92924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":92924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":92924}]},"public_addr":"172.21.15.16:6808/92924","cluster_addr":"172.21.15.16:6809/92924","heartbeat_back_addr":"172.21.15.16:6811/92924","heartbeat_front_addr":"172.21.15.16:6810/92924","state":["exists","up"]},{"osd":2,"uuid":"04ce138a-7d40-4c56-87e2-dd95b27be578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":92925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6805","nonce":92925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6807","nonce":92925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":92925}]},"public_addr":"172.21.15.16:6804/92925","cluster_addr":"172.21.15.16:6805/92925","heartbeat_back_addr":"172.21.15.16:6807/92925","heartbeat_front_addr":"172.21.15.16:6806/92925","state":["exists","up"]},{"osd":3,"uuid":"e1bfe8de-ea8b-4212-9de2-af8b9b83794e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":92937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":92937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":92937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":92937}]},"public_addr":"172.21.15.16:6812/92937","cluster_addr":"172.21.15.16:6813/92937","heartbeat_back_addr":"172.21.15.16:6815/92937","heartbeat_front_addr":"172.21.15.16:6814/92937","state":["exists","up"]},{"osd":4,"uuid":"d8d054e5-59ef-447b-97bc-0c81e47da65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":92926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6801","nonce":92926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6803","nonce":92926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":92926}]},"public_addr":"172.21.15.85:6800/92926","cluster_addr":"172.21.15.85:6801/92926","heartbeat_back_addr":"172.21.15.85:6803/92926","heartbeat_front_addr":"172.21.15.85:6802/92926","state":["exists","up"]},{"osd":5,"uuid":"02367b52-2c78-4576-b20f-71e603f35ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":92943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6813","nonce":92943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":92943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":92943}]},"public_addr":"172.21.15.85:6812/92943","cluster_addr":"172.21.15.85:6813/92943","heartbeat_back_addr":"172.21.15.85:6815/92943","heartbeat_front_addr":"172.21.15.85:6814/92943","state":["exists","up"]},{"osd":6,"uuid":"cd5c9527-4467-4828-9583-9851d083d931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":92928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6809","nonce":92928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6811","nonce":92928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":92928}]},"public_addr":"172.21.15.85:6808/92928","cluster_addr":"172.21.15.85:6809/92928","heartbeat_back_addr":"172.21.15.85:6811/92928","heartbeat_front_addr":"172.21.15.85:6810/92928","state":["exists","up"]},{"osd":7,"uuid":"918ee93e-d6c7-42c0-8112-f7cacc891ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":92927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6805","nonce":92927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6807","nonce":92927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":92927}]},"public_addr":"172.21.15.85:6804/92927","cluster_addr":"172.21.15.85:6805/92927","heartbeat_back_addr":"172.21.15.85:6807/92927","heartbeat_front_addr":"172.21.15.85:6806/92927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.127439+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.194972+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.133423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.210066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.179213+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.180101+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.189992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.138329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:22:06.442 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T05:22:06.443 INFO:tasks.ceph:Creating RBD pool 2023-01-25T05:22:06.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T05:22:07.016 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2023-01-25T05:22:07.017 INFO:tasks.ceph.osd.6.smithi085.stderr:2023-01-25T05:22:07.015+0000 7f23ff53d700 -1 osd.6 13 handle_osd_map injecting map crc failure 2023-01-25T05:22:07.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T05:22:07.231 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:07.230+0000 7fe826993700 -1 osd.5 13 handle_osd_map injecting map crc failure 2023-01-25T05:22:08.071 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T05:22:08.070+0000 7f9edb61f700 -1 osd.1 15 handle_osd_map injecting map crc failure 2023-01-25T05:22:08.073 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:08.072+0000 7fe826993700 -1 osd.5 15 handle_osd_map injecting map crc failure 2023-01-25T05:22:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T05:22:08.102 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T05:22:08.102 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T05:22:08.103 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T05:22:08.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T05:22:08.104 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T05:22:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:08.506 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":92312}]},"active_addr":"172.21.15.85:6816/92312","active_change":"2023-01-25T05:22:00.971645+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.85:0","nonce":4189736187}]},{"addrvec":[{"type":"v2","addr":"172.21.15.85:0","nonce":56531957}]},{"addrvec":[{"type":"v2","addr":"172.21.15.85:0","nonce":338391171}]}]}} 2023-01-25T05:22:08.510 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T05:22:08.510 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T05:22:08.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:22:08.838 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:08.839 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e","created":"2023-01-25T05:21:55.524643+0000","modified":"2023-01-25T05:22:08.011472+0000","last_up_change":"2023-01-25T05:22:04.995341+0000","last_in_change":"2023-01-25T05:21:57.989718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:22:05.040275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:22:06.797683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"88c5b9c1-6971-40ba-a114-73b6f37cb2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":92923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6801","nonce":92923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6803","nonce":92923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":92923}]},"public_addr":"172.21.15.16:6800/92923","cluster_addr":"172.21.15.16:6801/92923","heartbeat_back_addr":"172.21.15.16:6803/92923","heartbeat_front_addr":"172.21.15.16:6802/92923","state":["exists","up"]},{"osd":1,"uuid":"c6febc44-a227-44be-913a-e47a15762932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":92924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":92924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":92924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":92924}]},"public_addr":"172.21.15.16:6808/92924","cluster_addr":"172.21.15.16:6809/92924","heartbeat_back_addr":"172.21.15.16:6811/92924","heartbeat_front_addr":"172.21.15.16:6810/92924","state":["exists","up"]},{"osd":2,"uuid":"04ce138a-7d40-4c56-87e2-dd95b27be578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":92925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6805","nonce":92925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6807","nonce":92925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":92925}]},"public_addr":"172.21.15.16:6804/92925","cluster_addr":"172.21.15.16:6805/92925","heartbeat_back_addr":"172.21.15.16:6807/92925","heartbeat_front_addr":"172.21.15.16:6806/92925","state":["exists","up"]},{"osd":3,"uuid":"e1bfe8de-ea8b-4212-9de2-af8b9b83794e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":92937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":92937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":92937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":92937}]},"public_addr":"172.21.15.16:6812/92937","cluster_addr":"172.21.15.16:6813/92937","heartbeat_back_addr":"172.21.15.16:6815/92937","heartbeat_front_addr":"172.21.15.16:6814/92937","state":["exists","up"]},{"osd":4,"uuid":"d8d054e5-59ef-447b-97bc-0c81e47da65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":92926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6801","nonce":92926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6803","nonce":92926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":92926}]},"public_addr":"172.21.15.85:6800/92926","cluster_addr":"172.21.15.85:6801/92926","heartbeat_back_addr":"172.21.15.85:6803/92926","heartbeat_front_addr":"172.21.15.85:6802/92926","state":["exists","up"]},{"osd":5,"uuid":"02367b52-2c78-4576-b20f-71e603f35ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":92943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6813","nonce":92943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":92943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":92943}]},"public_addr":"172.21.15.85:6812/92943","cluster_addr":"172.21.15.85:6813/92943","heartbeat_back_addr":"172.21.15.85:6815/92943","heartbeat_front_addr":"172.21.15.85:6814/92943","state":["exists","up"]},{"osd":6,"uuid":"cd5c9527-4467-4828-9583-9851d083d931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":92928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6809","nonce":92928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6811","nonce":92928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":92928}]},"public_addr":"172.21.15.85:6808/92928","cluster_addr":"172.21.15.85:6809/92928","heartbeat_back_addr":"172.21.15.85:6811/92928","heartbeat_front_addr":"172.21.15.85:6810/92928","state":["exists","up"]},{"osd":7,"uuid":"918ee93e-d6c7-42c0-8112-f7cacc891ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":92927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6805","nonce":92927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6807","nonce":92927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":92927}]},"public_addr":"172.21.15.85:6804/92927","cluster_addr":"172.21.15.85:6805/92927","heartbeat_back_addr":"172.21.15.85:6807/92927","heartbeat_front_addr":"172.21.15.85:6806/92927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.127439+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.194972+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.133423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.210066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.179213+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.180101+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.189992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.138329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:22:08.853 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T05:22:08.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:22:09.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:09.187 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"fsid":"ce555a82-e9e2-45a7-bc6b-b2cccbe1ac5e","created":"2023-01-25T05:21:55.524643+0000","modified":"2023-01-25T05:22:09.070088+0000","last_up_change":"2023-01-25T05:22:04.995341+0000","last_in_change":"2023-01-25T05:21:57.989718+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:22:05.040275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:22:06.797683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"88c5b9c1-6971-40ba-a114-73b6f37cb2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":92923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6801","nonce":92923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6803","nonce":92923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":92923}]},"public_addr":"172.21.15.16:6800/92923","cluster_addr":"172.21.15.16:6801/92923","heartbeat_back_addr":"172.21.15.16:6803/92923","heartbeat_front_addr":"172.21.15.16:6802/92923","state":["exists","up"]},{"osd":1,"uuid":"c6febc44-a227-44be-913a-e47a15762932","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":92924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":92924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":92924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":92924}]},"public_addr":"172.21.15.16:6808/92924","cluster_addr":"172.21.15.16:6809/92924","heartbeat_back_addr":"172.21.15.16:6811/92924","heartbeat_front_addr":"172.21.15.16:6810/92924","state":["exists","up"]},{"osd":2,"uuid":"04ce138a-7d40-4c56-87e2-dd95b27be578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":92925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6805","nonce":92925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6807","nonce":92925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":92925}]},"public_addr":"172.21.15.16:6804/92925","cluster_addr":"172.21.15.16:6805/92925","heartbeat_back_addr":"172.21.15.16:6807/92925","heartbeat_front_addr":"172.21.15.16:6806/92925","state":["exists","up"]},{"osd":3,"uuid":"e1bfe8de-ea8b-4212-9de2-af8b9b83794e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":92937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":92937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":92937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":92937}]},"public_addr":"172.21.15.16:6812/92937","cluster_addr":"172.21.15.16:6813/92937","heartbeat_back_addr":"172.21.15.16:6815/92937","heartbeat_front_addr":"172.21.15.16:6814/92937","state":["exists","up"]},{"osd":4,"uuid":"d8d054e5-59ef-447b-97bc-0c81e47da65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":92926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6801","nonce":92926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6803","nonce":92926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":92926}]},"public_addr":"172.21.15.85:6800/92926","cluster_addr":"172.21.15.85:6801/92926","heartbeat_back_addr":"172.21.15.85:6803/92926","heartbeat_front_addr":"172.21.15.85:6802/92926","state":["exists","up"]},{"osd":5,"uuid":"02367b52-2c78-4576-b20f-71e603f35ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":92943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6813","nonce":92943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":92943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":92943}]},"public_addr":"172.21.15.85:6812/92943","cluster_addr":"172.21.15.85:6813/92943","heartbeat_back_addr":"172.21.15.85:6815/92943","heartbeat_front_addr":"172.21.15.85:6814/92943","state":["exists","up"]},{"osd":6,"uuid":"cd5c9527-4467-4828-9583-9851d083d931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":92928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6809","nonce":92928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6811","nonce":92928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":92928}]},"public_addr":"172.21.15.85:6808/92928","cluster_addr":"172.21.15.85:6809/92928","heartbeat_back_addr":"172.21.15.85:6811/92928","heartbeat_front_addr":"172.21.15.85:6810/92928","state":["exists","up"]},{"osd":7,"uuid":"918ee93e-d6c7-42c0-8112-f7cacc891ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":92927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6805","nonce":92927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6807","nonce":92927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":92927}]},"public_addr":"172.21.15.85:6804/92927","cluster_addr":"172.21.15.85:6805/92927","heartbeat_back_addr":"172.21.15.85:6807/92927","heartbeat_front_addr":"172.21.15.85:6806/92927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.127439+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.194972+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.133423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.210066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.179213+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.180101+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.189992+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:22:03.138329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:22:09.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T05:22:09.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T05:22:09.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T05:22:09.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T05:22:09.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T05:22:09.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T05:22:09.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T05:22:09.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T05:22:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:22:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:22:09.501 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:22:09.505 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:22:09.518 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:22:09.520 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:22:09.522 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:22:09.530 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:09.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:22:10.106 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-25T05:22:10.139 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.143 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.151 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-25T05:22:10.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-01-25T05:22:10.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-25T05:22:10.172 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-01-25T05:22:10.211 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.214 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-25T05:22:10.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-25T05:22:10.233 INFO:teuthology.orchestra.run.smithi016.stdout:55834574850 2023-01-25T05:22:10.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T05:22:11.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:22:11.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:22:11.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:22:11.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:22:11.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:22:11.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:22:11.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:22:11.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:22:11.710 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2023-01-25T05:22:11.732 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.779 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.779 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2023-01-25T05:22:11.793 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.798 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2023-01-25T05:22:11.803 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-25T05:22:11.820 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.856 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.862 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-25T05:22:11.872 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-25T05:22:11.882 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.897 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.906 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-01-25T05:22:11.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-25T05:22:11.911 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2023-01-25T05:22:11.921 DEBUG:teuthology.parallel:result is None 2023-01-25T05:22:11.921 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T05:22:11.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:22:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:12.288 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T05:22:12.300 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T05:22:10.977755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":41296,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":40128,"kb_avail":754933424,"statfs":{"total":773094113280,"available":773051826176,"internally_reserved":0,"allocated":1073152,"data_stored":1374142,"data_compressed":16556,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41091072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.149632+0000","last_change":"2023-01-25T05:22:08.079154+0000","last_active":"2023-01-25T05:22:09.149632+0000","last_peered":"2023-01-25T05:22:09.149632+0000","last_clean":"2023-01-25T05:22:09.149632+0000","last_became_active":"2023-01-25T05:22:08.078869+0000","last_became_peered":"2023-01-25T05:22:08.078869+0000","last_unstale":"2023-01-25T05:22:09.149632+0000","last_undegraded":"2023-01-25T05:22:09.149632+0000","last_fullsized":"2023-01-25T05:22:09.149632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:02:56.386644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:22:08.079663+0000","last_change":"2023-01-25T05:22:08.078965+0000","last_active":"2023-01-25T05:22:08.079663+0000","last_peered":"2023-01-25T05:22:08.079663+0000","last_clean":"2023-01-25T05:22:08.079663+0000","last_became_active":"2023-01-25T05:22:08.078666+0000","last_became_peered":"2023-01-25T05:22:08.078666+0000","last_unstale":"2023-01-25T05:22:08.079663+0000","last_undegraded":"2023-01-25T05:22:08.079663+0000","last_fullsized":"2023-01-25T05:22:08.079663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:25:03.396290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143908+0000","last_change":"2023-01-25T05:22:08.077623+0000","last_active":"2023-01-25T05:22:09.143908+0000","last_peered":"2023-01-25T05:22:09.143908+0000","last_clean":"2023-01-25T05:22:09.143908+0000","last_became_active":"2023-01-25T05:22:08.077349+0000","last_became_peered":"2023-01-25T05:22:08.077349+0000","last_unstale":"2023-01-25T05:22:09.143908+0000","last_undegraded":"2023-01-25T05:22:09.143908+0000","last_fullsized":"2023-01-25T05:22:09.143908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:19:43.134163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143893+0000","last_change":"2023-01-25T05:22:08.077788+0000","last_active":"2023-01-25T05:22:09.143893+0000","last_peered":"2023-01-25T05:22:09.143893+0000","last_clean":"2023-01-25T05:22:09.143893+0000","last_became_active":"2023-01-25T05:22:08.077513+0000","last_became_peered":"2023-01-25T05:22:08.077513+0000","last_unstale":"2023-01-25T05:22:09.143893+0000","last_undegraded":"2023-01-25T05:22:09.143893+0000","last_fullsized":"2023-01-25T05:22:09.143893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:47:51.533695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143887+0000","last_change":"2023-01-25T05:22:07.027369+0000","last_active":"2023-01-25T05:22:09.143887+0000","last_peered":"2023-01-25T05:22:09.143887+0000","last_clean":"2023-01-25T05:22:09.143887+0000","last_became_active":"2023-01-25T05:22:07.026475+0000","last_became_peered":"2023-01-25T05:22:07.026475+0000","last_unstale":"2023-01-25T05:22:09.143887+0000","last_undegraded":"2023-01-25T05:22:09.143887+0000","last_fullsized":"2023-01-25T05:22:09.143887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:06.002083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:06.002083+0000","last_clean_scrub_stamp":"2023-01-25T05:22:06.002083+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:28:44.577344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.184170+0000","last_change":"2023-01-25T05:22:08.078797+0000","last_active":"2023-01-25T05:22:09.184170+0000","last_peered":"2023-01-25T05:22:09.184170+0000","last_clean":"2023-01-25T05:22:09.184170+0000","last_became_active":"2023-01-25T05:22:08.078514+0000","last_became_peered":"2023-01-25T05:22:08.078514+0000","last_unstale":"2023-01-25T05:22:09.184170+0000","last_undegraded":"2023-01-25T05:22:09.184170+0000","last_fullsized":"2023-01-25T05:22:09.184170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:37:34.756573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.085374+0000","last_change":"2023-01-25T05:22:08.085129+0000","last_active":"2023-01-25T05:22:09.085374+0000","last_peered":"2023-01-25T05:22:09.085374+0000","last_clean":"2023-01-25T05:22:09.085374+0000","last_became_active":"2023-01-25T05:22:08.084837+0000","last_became_peered":"2023-01-25T05:22:08.084837+0000","last_unstale":"2023-01-25T05:22:09.085374+0000","last_undegraded":"2023-01-25T05:22:09.085374+0000","last_fullsized":"2023-01-25T05:22:09.085374+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:22:30.733666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.184190+0000","last_change":"2023-01-25T05:22:08.079007+0000","last_active":"2023-01-25T05:22:09.184190+0000","last_peered":"2023-01-25T05:22:09.184190+0000","last_clean":"2023-01-25T05:22:09.184190+0000","last_became_active":"2023-01-25T05:22:08.078456+0000","last_became_peered":"2023-01-25T05:22:08.078456+0000","last_unstale":"2023-01-25T05:22:09.184190+0000","last_undegraded":"2023-01-25T05:22:09.184190+0000","last_fullsized":"2023-01-25T05:22:09.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:29:03.966813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:22:08.077203+0000","last_change":"2023-01-25T05:22:08.076079+0000","last_active":"2023-01-25T05:22:08.077203+0000","last_peered":"2023-01-25T05:22:08.077203+0000","last_clean":"2023-01-25T05:22:08.077203+0000","last_became_active":"2023-01-25T05:22:08.075793+0000","last_became_peered":"2023-01-25T05:22:08.075793+0000","last_unstale":"2023-01-25T05:22:08.077203+0000","last_undegraded":"2023-01-25T05:22:08.077203+0000","last_fullsized":"2023-01-25T05:22:08.077203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:23:30.747669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":16556,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5560,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366280,"statfs":{"total":96636764160,"available":96631070720,"internally_reserved":0,"allocated":151552,"data_stored":566238,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":37870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":36878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5596,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366244,"statfs":{"total":96636764160,"available":96631033856,"internally_reserved":0,"allocated":188416,"data_stored":569530,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4980,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366860,"statfs":{"total":96636764160,"available":96631664640,"internally_reserved":0,"allocated":110592,"data_stored":37466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":45202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:22:12.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:22:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T05:22:12.640 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T05:22:10.977755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":41296,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":40128,"kb_avail":754933424,"statfs":{"total":773094113280,"available":773051826176,"internally_reserved":0,"allocated":1073152,"data_stored":1374142,"data_compressed":16556,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41091072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.149632+0000","last_change":"2023-01-25T05:22:08.079154+0000","last_active":"2023-01-25T05:22:09.149632+0000","last_peered":"2023-01-25T05:22:09.149632+0000","last_clean":"2023-01-25T05:22:09.149632+0000","last_became_active":"2023-01-25T05:22:08.078869+0000","last_became_peered":"2023-01-25T05:22:08.078869+0000","last_unstale":"2023-01-25T05:22:09.149632+0000","last_undegraded":"2023-01-25T05:22:09.149632+0000","last_fullsized":"2023-01-25T05:22:09.149632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:02:56.386644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:22:08.079663+0000","last_change":"2023-01-25T05:22:08.078965+0000","last_active":"2023-01-25T05:22:08.079663+0000","last_peered":"2023-01-25T05:22:08.079663+0000","last_clean":"2023-01-25T05:22:08.079663+0000","last_became_active":"2023-01-25T05:22:08.078666+0000","last_became_peered":"2023-01-25T05:22:08.078666+0000","last_unstale":"2023-01-25T05:22:08.079663+0000","last_undegraded":"2023-01-25T05:22:08.079663+0000","last_fullsized":"2023-01-25T05:22:08.079663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:25:03.396290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143908+0000","last_change":"2023-01-25T05:22:08.077623+0000","last_active":"2023-01-25T05:22:09.143908+0000","last_peered":"2023-01-25T05:22:09.143908+0000","last_clean":"2023-01-25T05:22:09.143908+0000","last_became_active":"2023-01-25T05:22:08.077349+0000","last_became_peered":"2023-01-25T05:22:08.077349+0000","last_unstale":"2023-01-25T05:22:09.143908+0000","last_undegraded":"2023-01-25T05:22:09.143908+0000","last_fullsized":"2023-01-25T05:22:09.143908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:19:43.134163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143893+0000","last_change":"2023-01-25T05:22:08.077788+0000","last_active":"2023-01-25T05:22:09.143893+0000","last_peered":"2023-01-25T05:22:09.143893+0000","last_clean":"2023-01-25T05:22:09.143893+0000","last_became_active":"2023-01-25T05:22:08.077513+0000","last_became_peered":"2023-01-25T05:22:08.077513+0000","last_unstale":"2023-01-25T05:22:09.143893+0000","last_undegraded":"2023-01-25T05:22:09.143893+0000","last_fullsized":"2023-01-25T05:22:09.143893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:47:51.533695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.143887+0000","last_change":"2023-01-25T05:22:07.027369+0000","last_active":"2023-01-25T05:22:09.143887+0000","last_peered":"2023-01-25T05:22:09.143887+0000","last_clean":"2023-01-25T05:22:09.143887+0000","last_became_active":"2023-01-25T05:22:07.026475+0000","last_became_peered":"2023-01-25T05:22:07.026475+0000","last_unstale":"2023-01-25T05:22:09.143887+0000","last_undegraded":"2023-01-25T05:22:09.143887+0000","last_fullsized":"2023-01-25T05:22:09.143887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:06.002083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:06.002083+0000","last_clean_scrub_stamp":"2023-01-25T05:22:06.002083+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:28:44.577344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.184170+0000","last_change":"2023-01-25T05:22:08.078797+0000","last_active":"2023-01-25T05:22:09.184170+0000","last_peered":"2023-01-25T05:22:09.184170+0000","last_clean":"2023-01-25T05:22:09.184170+0000","last_became_active":"2023-01-25T05:22:08.078514+0000","last_became_peered":"2023-01-25T05:22:08.078514+0000","last_unstale":"2023-01-25T05:22:09.184170+0000","last_undegraded":"2023-01-25T05:22:09.184170+0000","last_fullsized":"2023-01-25T05:22:09.184170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:37:34.756573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.085374+0000","last_change":"2023-01-25T05:22:08.085129+0000","last_active":"2023-01-25T05:22:09.085374+0000","last_peered":"2023-01-25T05:22:09.085374+0000","last_clean":"2023-01-25T05:22:09.085374+0000","last_became_active":"2023-01-25T05:22:08.084837+0000","last_became_peered":"2023-01-25T05:22:08.084837+0000","last_unstale":"2023-01-25T05:22:09.085374+0000","last_undegraded":"2023-01-25T05:22:09.085374+0000","last_fullsized":"2023-01-25T05:22:09.085374+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:22:30.733666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:22:09.184190+0000","last_change":"2023-01-25T05:22:08.079007+0000","last_active":"2023-01-25T05:22:09.184190+0000","last_peered":"2023-01-25T05:22:09.184190+0000","last_clean":"2023-01-25T05:22:09.184190+0000","last_became_active":"2023-01-25T05:22:08.078456+0000","last_became_peered":"2023-01-25T05:22:08.078456+0000","last_unstale":"2023-01-25T05:22:09.184190+0000","last_undegraded":"2023-01-25T05:22:09.184190+0000","last_fullsized":"2023-01-25T05:22:09.184190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:29:03.966813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:22:08.077203+0000","last_change":"2023-01-25T05:22:08.076079+0000","last_active":"2023-01-25T05:22:08.077203+0000","last_peered":"2023-01-25T05:22:08.077203+0000","last_clean":"2023-01-25T05:22:08.077203+0000","last_became_active":"2023-01-25T05:22:08.075793+0000","last_became_peered":"2023-01-25T05:22:08.075793+0000","last_unstale":"2023-01-25T05:22:08.077203+0000","last_undegraded":"2023-01-25T05:22:08.077203+0000","last_fullsized":"2023-01-25T05:22:08.077203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:22:07.007499+0000","last_clean_scrub_stamp":"2023-01-25T05:22:07.007499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:23:30.747669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":16556,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5560,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366280,"statfs":{"total":96636764160,"available":96631070720,"internally_reserved":0,"allocated":151552,"data_stored":566238,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":37870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":36878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5596,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366244,"statfs":{"total":96636764160,"available":96631033856,"internally_reserved":0,"allocated":188416,"data_stored":569530,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4980,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366860,"statfs":{"total":96636764160,"available":96631664640,"internally_reserved":0,"allocated":110592,"data_stored":37466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":45202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8278,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:22:12.641 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T05:22:12.642 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T05:22:12.642 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T05:22:12.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T05:22:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T05:22:13.012 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T05:22:13.026 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T05:22:13.026 INFO:teuthology.run_tasks:Running task exec... 2023-01-25T05:22:13.037 INFO:teuthology.task.exec:Executing custom commands... 2023-01-25T05:22:13.037 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:22:13.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2023-01-25T05:22:13.365 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-01-25T05:22:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "active": true, 2023-01-25T05:22:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "last_optimize_duration": "0:00:00.000819", 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "last_optimize_started": "Wed Jan 25 05:22:00 2023", 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "upmap", 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "no_optimization_needed": false, 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "optimize_result": "No pools available", 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "plans": [] 2023-01-25T05:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-01-25T05:22:13.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2023-01-25T05:22:14.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2023-01-25T05:22:14.668 INFO:teuthology.run_tasks:Running task exec... 2023-01-25T05:22:14.677 INFO:teuthology.task.exec:Executing custom commands... 2023-01-25T05:22:14.677 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T05:22:14.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create base 4' 2023-01-25T05:22:14.915 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:14.914+0000 7f749b4db700 -1 osd.0 17 handle_osd_map injecting map crc failure 2023-01-25T05:22:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'base' created 2023-01-25T05:22:15.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool application enable base rados' 2023-01-25T05:22:16.310 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T05:22:16.309+0000 7f2425a1f700 -1 osd.2 19 handle_osd_map injecting map crc failure 2023-01-25T05:22:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rados' on pool 'base' 2023-01-25T05:22:16.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2023-01-25T05:22:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cache' created 2023-01-25T05:22:17.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add base cache' 2023-01-25T05:22:17.358 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:17.357+0000 7f749b4db700 -1 osd.0 20 handle_osd_map injecting map crc failure 2023-01-25T05:22:17.358 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:22:17.357+0000 7f15b7a56700 -1 osd.3 20 handle_osd_map injecting map crc failure 2023-01-25T05:22:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cache' is now (or already was) a tier of 'base' 2023-01-25T05:22:18.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2023-01-25T05:22:19.323 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T05:22:19.322+0000 7f15b9a5a700 -1 osd.3 22 handle_osd_map injecting map crc failure 2023-01-25T05:22:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:set cache-mode for pool 'cache' to writeback 2023-01-25T05:22:19.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay base cache' 2023-01-25T05:22:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:overlay for 'base' is now (or already was) 'cache' 2023-01-25T05:22:20.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2023-01-25T05:22:21.341 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:21.341+0000 7fe826993700 -1 osd.5 24 handle_osd_map injecting map crc failure 2023-01-25T05:22:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 hit_set_type to bloom 2023-01-25T05:22:21.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2023-01-25T05:22:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 hit_set_count to 8 2023-01-25T05:22:22.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 3600' 2023-01-25T05:22:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 hit_set_period to 3600 2023-01-25T05:22:23.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2023-01-25T05:22:23.769 INFO:tasks.ceph.osd.5.smithi085.stderr:2023-01-25T05:22:23.768+0000 7fe82498f700 -1 osd.5 26 handle_osd_map injecting map crc failure 2023-01-25T05:22:24.353 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T05:22:24.353+0000 7f749d4df700 -1 osd.0 27 handle_osd_map injecting map crc failure 2023-01-25T05:22:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 target_max_objects to 250 2023-01-25T05:22:24.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache min_read_recency_for_promote 2' 2023-01-25T05:22:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 min_read_recency_for_promote to 2 2023-01-25T05:22:25.378 INFO:teuthology.run_tasks:Running task rados... 2023-01-25T05:22:25.390 INFO:tasks.rados:Beginning rados... 2023-01-25T05:22:25.390 DEBUG:teuthology.run_tasks:Unwinding manager rados 2023-01-25T05:22:25.406 INFO:tasks.rados:joining rados 2023-01-25T05:22:25.406 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2023-01-25T05:22:25.406 INFO:tasks.rados:starting run 0 out of 1 2023-01-25T05:22:25.407 DEBUG:teuthology.orchestra.run.smithi016:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --balance-reads --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op cache_flush 50 --op cache_try_flush 50 --op cache_evict 50 --op write_excl 50 --pool base 2023-01-25T05:22:25.440 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight read -> 100 2023-01-25T05:22:25.441 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight write -> 50 2023-01-25T05:22:25.441 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight delete -> 50 2023-01-25T05:22:25.441 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight snap_create -> 50 2023-01-25T05:22:25.441 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight snap_remove -> 50 2023-01-25T05:22:25.441 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight rollback -> 50 2023-01-25T05:22:25.442 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight copy_from -> 50 2023-01-25T05:22:25.442 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight cache_flush -> 50 2023-01-25T05:22:25.442 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight cache_try_flush -> 50 2023-01-25T05:22:25.442 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight cache_evict -> 50 2023-01-25T05:22:25.443 INFO:tasks.rados.rados.0.smithi016.stdout:adding op weight write_excl -> 50 2023-01-25T05:22:25.443 INFO:tasks.rados.rados.0.smithi016.stdout:ceph version 18.0.0-1959-gb0d25925 (b0d2592529629ab4f86fed5991e556a87dcda563) reef (dev) 2023-01-25T05:22:25.443 INFO:tasks.rados.rados.0.smithi016.stdout:Configuration: 2023-01-25T05:22:25.443 INFO:tasks.rados.rados.0.smithi016.stdout: Number of operations: 4000 2023-01-25T05:22:25.443 INFO:tasks.rados.rados.0.smithi016.stdout: Number of objects: 500 2023-01-25T05:22:25.444 INFO:tasks.rados.rados.0.smithi016.stdout: Max in flight operations: 16 2023-01-25T05:22:25.444 INFO:tasks.rados.rados.0.smithi016.stdout: Object size (in bytes): 4000000 2023-01-25T05:22:25.444 INFO:tasks.rados.rados.0.smithi016.stdout: Write stride min: 400000 2023-01-25T05:22:25.444 INFO:tasks.rados.rados.0.smithi016.stdout: Write stride max: 800000 2023-01-25T05:22:26.366 INFO:tasks.rados.rados.0.smithi016.stdout:1: write initial oid 1 2023-01-25T05:22:26.366 INFO:tasks.rados.rados.0.smithi016.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2023-01-25T05:22:26.368 INFO:tasks.ceph.osd.4.smithi085.stderr:2023-01-25T05:22:26.368+0000 7fbbca72d700 -1 osd.4 29 handle_osd_map injecting map crc failure 2023-01-25T05:22:26.370 INFO:tasks.rados.rados.0.smithi016.stdout:1: writing smithi01698494-1 from 416807 to 892056 tid 1 2023-01-25T05:22:26.377 INFO:tasks.rados.rados.0.smithi016.stdout:1: writing smithi01698494-1 from 1542129 to 2016807 tid 2 2023-01-25T05:22:26.377 INFO:tasks.rados.rados.0.smithi016.stdout:2: write initial oid 2 2023-01-25T05:22:26.377 INFO:tasks.rados.rados.0.smithi016.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2023-01-25T05:22:26.382 INFO:tasks.rados.rados.0.smithi016.stdout:2: writing smithi01698494-2 from 416807 to 892056 tid 1 2023-01-25T05:22:26.389 INFO:tasks.rados.rados.0.smithi016.stdout:2: writing smithi01698494-2 from 1542129 to 2016807 tid 2 2023-01-25T05:22:26.389 INFO:tasks.rados.rados.0.smithi016.stdout:3: write initial oid 3 2023-01-25T05:22:26.390 INFO:tasks.rados.rados.0.smithi016.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2023-01-25T05:22:26.395 INFO:tasks.rados.rados.0.smithi016.stdout:3: writing smithi01698494-3 from 433614 to 984112 tid 1 2023-01-25T05:22:26.401 INFO:tasks.rados.rados.0.smithi016.stdout:3: writing smithi01698494-3 from 1600611 to 2033614 tid 2 2023-01-25T05:22:26.401 INFO:tasks.rados.rados.0.smithi016.stdout:4: write initial oid 4 2023-01-25T05:22:26.402 INFO:tasks.rados.rados.0.smithi016.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2023-01-25T05:22:26.407 INFO:tasks.rados.rados.0.smithi016.stdout:4: writing smithi01698494-4 from 450421 to 1076168 tid 1 2023-01-25T05:22:26.414 INFO:tasks.rados.rados.0.smithi016.stdout:4: writing smithi01698494-4 from 1659093 to 2050421 tid 2 2023-01-25T05:22:26.414 INFO:tasks.rados.rados.0.smithi016.stdout:5: write initial oid 5 2023-01-25T05:22:26.414 INFO:tasks.rados.rados.0.smithi016.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2023-01-25T05:22:26.419 INFO:tasks.rados.rados.0.smithi016.stdout:5: writing smithi01698494-5 from 467228 to 1168224 tid 1 2023-01-25T05:22:26.424 INFO:tasks.rados.rados.0.smithi016.stdout:5: writing smithi01698494-5 from 1717575 to 2067228 tid 2 2023-01-25T05:22:26.424 INFO:tasks.rados.rados.0.smithi016.stdout:6: write initial oid 6 2023-01-25T05:22:26.425 INFO:tasks.rados.rados.0.smithi016.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2023-01-25T05:22:26.431 INFO:tasks.rados.rados.0.smithi016.stdout:6: writing smithi01698494-6 from 484035 to 1260280 tid 1 2023-01-25T05:22:26.436 INFO:tasks.rados.rados.0.smithi016.stdout:6: writing smithi01698494-6 from 2059704 to 2084035 tid 2 2023-01-25T05:22:26.436 INFO:tasks.rados.rados.0.smithi016.stdout:7: write initial oid 7 2023-01-25T05:22:26.436 INFO:tasks.rados.rados.0.smithi016.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2023-01-25T05:22:26.441 INFO:tasks.rados.rados.0.smithi016.stdout:7: writing smithi01698494-7 from 500842 to 952336 tid 1 2023-01-25T05:22:26.447 INFO:tasks.rados.rados.0.smithi016.stdout:7: writing smithi01698494-7 from 1718186 to 2100842 tid 2 2023-01-25T05:22:26.448 INFO:tasks.rados.rados.0.smithi016.stdout:8: write initial oid 8 2023-01-25T05:22:26.448 INFO:tasks.rados.rados.0.smithi016.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2023-01-25T05:22:26.454 INFO:tasks.rados.rados.0.smithi016.stdout:8: writing smithi01698494-8 from 517649 to 1044392 tid 1 2023-01-25T05:22:26.459 INFO:tasks.rados.rados.0.smithi016.stdout:8: writing smithi01698494-8 from 1776668 to 2117649 tid 2 2023-01-25T05:22:26.460 INFO:tasks.rados.rados.0.smithi016.stdout:9: write initial oid 9 2023-01-25T05:22:26.460 INFO:tasks.rados.rados.0.smithi016.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2023-01-25T05:22:26.466 INFO:tasks.rados.rados.0.smithi016.stdout:9: writing smithi01698494-9 from 534456 to 1252801 tid 1 2023-01-25T05:22:26.471 INFO:tasks.rados.rados.0.smithi016.stdout:9: writing smithi01698494-9 from 1951503 to 2134456 tid 2 2023-01-25T05:22:26.472 INFO:tasks.rados.rados.0.smithi016.stdout:10: write initial oid 10 2023-01-25T05:22:26.472 INFO:tasks.rados.rados.0.smithi016.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2023-01-25T05:22:26.479 INFO:tasks.rados.rados.0.smithi016.stdout:10: writing smithi01698494-10 from 551263 to 1344857 tid 1 2023-01-25T05:22:26.483 INFO:tasks.rados.rados.0.smithi016.stdout:10: writing smithi01698494-10 from 1893632 to 2151263 tid 2 2023-01-25T05:22:26.484 INFO:tasks.rados.rados.0.smithi016.stdout:11: write initial oid 11 2023-01-25T05:22:26.484 INFO:tasks.rados.rados.0.smithi016.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2023-01-25T05:22:26.489 INFO:tasks.rados.rados.0.smithi016.stdout:11: writing smithi01698494-11 from 568070 to 1036913 tid 1 2023-01-25T05:22:26.496 INFO:tasks.rados.rados.0.smithi016.stdout:11: writing smithi01698494-11 from 1552114 to 2168070 tid 2 2023-01-25T05:22:26.496 INFO:tasks.rados.rados.0.smithi016.stdout:12: write initial oid 12 2023-01-25T05:22:26.496 INFO:tasks.rados.rados.0.smithi016.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2023-01-25T05:22:26.502 INFO:tasks.rados.rados.0.smithi016.stdout:12: writing smithi01698494-12 from 584877 to 1128969 tid 1 2023-01-25T05:22:26.508 INFO:tasks.rados.rados.0.smithi016.stdout:12: writing smithi01698494-12 from 1610596 to 2172599 tid 2 2023-01-25T05:22:26.508 INFO:tasks.rados.rados.0.smithi016.stdout:13: write initial oid 13 2023-01-25T05:22:26.508 INFO:tasks.rados.rados.0.smithi016.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2023-01-25T05:22:26.515 INFO:tasks.rados.rados.0.smithi016.stdout:13: writing smithi01698494-13 from 601684 to 1221025 tid 1 2023-01-25T05:22:26.520 INFO:tasks.rados.rados.0.smithi016.stdout:13: writing smithi01698494-13 from 1669078 to 2201684 tid 2 2023-01-25T05:22:26.521 INFO:tasks.rados.rados.0.smithi016.stdout:14: write initial oid 14 2023-01-25T05:22:26.521 INFO:tasks.rados.rados.0.smithi016.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2023-01-25T05:22:26.527 INFO:tasks.rados.rados.0.smithi016.stdout:14: writing smithi01698494-14 from 618491 to 1313081 tid 1 2023-01-25T05:22:26.533 INFO:tasks.rados.rados.0.smithi016.stdout:14: writing smithi01698494-14 from 2011207 to 2218491 tid 2 2023-01-25T05:22:26.533 INFO:tasks.rados.rados.0.smithi016.stdout:15: write initial oid 15 2023-01-25T05:22:26.533 INFO:tasks.rados.rados.0.smithi016.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2023-01-25T05:22:26.540 INFO:tasks.rados.rados.0.smithi016.stdout:15: writing smithi01698494-15 from 635298 to 1405137 tid 1 2023-01-25T05:22:26.545 INFO:tasks.rados.rados.0.smithi016.stdout:15: writing smithi01698494-15 from 2069689 to 2235298 tid 2 2023-01-25T05:22:26.546 INFO:tasks.rados.rados.0.smithi016.stdout:16: write initial oid 16 2023-01-25T05:22:26.546 INFO:tasks.rados.rados.0.smithi016.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2023-01-25T05:22:26.551 INFO:tasks.rados.rados.0.smithi016.stdout:16: writing smithi01698494-16 from 652105 to 1097193 tid 1 2023-01-25T05:22:26.559 INFO:tasks.rados.rados.0.smithi016.stdout:16: writing smithi01698494-16 from 1728171 to 2181159 tid 2 2023-01-25T05:22:26.560 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.560 INFO:tasks.rados.rados.0.smithi016.stdout:1: finishing write tid 1 to smithi01698494-1 2023-01-25T05:22:26.560 INFO:tasks.rados.rados.0.smithi016.stdout:1: finishing write tid 2 to smithi01698494-1 2023-01-25T05:22:26.560 INFO:tasks.rados.rados.0.smithi016.stdout:1: finishing write tid 3 to smithi01698494-1 2023-01-25T05:22:26.561 INFO:tasks.rados.rados.0.smithi016.stdout:1: finishing write tid 4 to smithi01698494-1 2023-01-25T05:22:26.561 INFO:tasks.rados.rados.0.smithi016.stdout:1: oid 1 updating version 0 to 1 2023-01-25T05:22:26.561 INFO:tasks.rados.rados.0.smithi016.stdout:1: oid 1 updating version 1 to 2 2023-01-25T05:22:26.561 INFO:tasks.rados.rados.0.smithi016.stdout:1: oid 1 updating version 2 to 3 2023-01-25T05:22:26.561 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2023-01-25T05:22:26.562 INFO:tasks.rados.rados.0.smithi016.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2023-01-25T05:22:26.562 INFO:tasks.rados.rados.0.smithi016.stdout:2: finishing write tid 1 to smithi01698494-2 2023-01-25T05:22:26.562 INFO:tasks.rados.rados.0.smithi016.stdout:1: done (15 left) 2023-01-25T05:22:26.562 INFO:tasks.rados.rados.0.smithi016.stdout:17: write initial oid 17 2023-01-25T05:22:26.562 INFO:tasks.rados.rados.0.smithi016.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2023-01-25T05:22:26.565 INFO:tasks.rados.rados.0.smithi016.stdout:17: writing smithi01698494-17 from 668912 to 1305602 tid 1 2023-01-25T05:22:26.571 INFO:tasks.rados.rados.0.smithi016.stdout:17: writing smithi01698494-17 from 1903006 to 2268912 tid 2 2023-01-25T05:22:26.571 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.571 INFO:tasks.rados.rados.0.smithi016.stdout:2: finishing write tid 2 to smithi01698494-2 2023-01-25T05:22:26.571 INFO:tasks.rados.rados.0.smithi016.stdout:2: finishing write tid 3 to smithi01698494-2 2023-01-25T05:22:26.571 INFO:tasks.rados.rados.0.smithi016.stdout:2: finishing write tid 4 to smithi01698494-2 2023-01-25T05:22:26.572 INFO:tasks.rados.rados.0.smithi016.stdout:2: oid 2 updating version 0 to 1 2023-01-25T05:22:26.572 INFO:tasks.rados.rados.0.smithi016.stdout:2: oid 2 updating version 1 to 2 2023-01-25T05:22:26.572 INFO:tasks.rados.rados.0.smithi016.stdout:2: oid 2 updating version 2 to 3 2023-01-25T05:22:26.572 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2023-01-25T05:22:26.572 INFO:tasks.rados.rados.0.smithi016.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2023-01-25T05:22:26.573 INFO:tasks.rados.rados.0.smithi016.stdout:4: finishing write tid 1 to smithi01698494-4 2023-01-25T05:22:26.573 INFO:tasks.rados.rados.0.smithi016.stdout:2: done (15 left) 2023-01-25T05:22:26.573 INFO:tasks.rados.rados.0.smithi016.stdout:18: write initial oid 18 2023-01-25T05:22:26.573 INFO:tasks.rados.rados.0.smithi016.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2023-01-25T05:22:26.578 INFO:tasks.rados.rados.0.smithi016.stdout:18: writing smithi01698494-18 from 685719 to 1397658 tid 1 2023-01-25T05:22:26.584 INFO:tasks.rados.rados.0.smithi016.stdout:18: writing smithi01698494-18 from 1845135 to 2285719 tid 2 2023-01-25T05:22:26.584 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.584 INFO:tasks.rados.rados.0.smithi016.stdout:4: finishing write tid 2 to smithi01698494-4 2023-01-25T05:22:26.584 INFO:tasks.rados.rados.0.smithi016.stdout:4: finishing write tid 3 to smithi01698494-4 2023-01-25T05:22:26.584 INFO:tasks.rados.rados.0.smithi016.stdout:4: finishing write tid 4 to smithi01698494-4 2023-01-25T05:22:26.585 INFO:tasks.rados.rados.0.smithi016.stdout:4: oid 4 updating version 0 to 5 2023-01-25T05:22:26.585 INFO:tasks.rados.rados.0.smithi016.stdout:4: oid 4 version 5 is already newer than 2 2023-01-25T05:22:26.585 INFO:tasks.rados.rados.0.smithi016.stdout:4: oid 4 updating version 5 to 6 2023-01-25T05:22:26.586 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 4 v 6 (ObjNum 3 snap 0 seq_num 3) dirty exists 2023-01-25T05:22:26.586 INFO:tasks.rados.rados.0.smithi016.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2023-01-25T05:22:26.586 INFO:tasks.rados.rados.0.smithi016.stdout:5: finishing write tid 1 to smithi01698494-5 2023-01-25T05:22:26.586 INFO:tasks.rados.rados.0.smithi016.stdout:5: finishing write tid 2 to smithi01698494-5 2023-01-25T05:22:26.586 INFO:tasks.rados.rados.0.smithi016.stdout:5: finishing write tid 3 to smithi01698494-5 2023-01-25T05:22:26.587 INFO:tasks.rados.rados.0.smithi016.stdout:4: done (15 left) 2023-01-25T05:22:26.587 INFO:tasks.rados.rados.0.smithi016.stdout:19: write initial oid 19 2023-01-25T05:22:26.587 INFO:tasks.rados.rados.0.smithi016.stdout:5: finishing write tid 4 to smithi01698494-5 2023-01-25T05:22:26.587 INFO:tasks.rados.rados.0.smithi016.stdout:5: oid 5 updating version 0 to 2 2023-01-25T05:22:26.587 INFO:tasks.rados.rados.0.smithi016.stdout:5: oid 5 version 2 is already newer than 1 2023-01-25T05:22:26.588 INFO:tasks.rados.rados.0.smithi016.stdout:5: oid 5 updating version 2 to 3 2023-01-25T05:22:26.588 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2023-01-25T05:22:26.588 INFO:tasks.rados.rados.0.smithi016.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2023-01-25T05:22:26.588 INFO:tasks.rados.rados.0.smithi016.stdout:8: finishing write tid 1 to smithi01698494-8 2023-01-25T05:22:26.589 INFO:tasks.rados.rados.0.smithi016.stdout:8: finishing write tid 2 to smithi01698494-8 2023-01-25T05:22:26.589 INFO:tasks.rados.rados.0.smithi016.stdout:8: finishing write tid 3 to smithi01698494-8 2023-01-25T05:22:26.589 INFO:tasks.rados.rados.0.smithi016.stdout:8: finishing write tid 4 to smithi01698494-8 2023-01-25T05:22:26.589 INFO:tasks.rados.rados.0.smithi016.stdout:8: oid 8 updating version 0 to 5 2023-01-25T05:22:26.589 INFO:tasks.rados.rados.0.smithi016.stdout:8: oid 8 version 5 is already newer than 2 2023-01-25T05:22:26.590 INFO:tasks.rados.rados.0.smithi016.stdout:8: oid 8 updating version 5 to 6 2023-01-25T05:22:26.590 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2023-01-25T05:22:26.590 INFO:tasks.rados.rados.0.smithi016.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2023-01-25T05:22:26.590 INFO:tasks.rados.rados.0.smithi016.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2023-01-25T05:22:26.591 INFO:tasks.rados.rados.0.smithi016.stdout:19: writing smithi01698494-19 from 702526 to 1489714 tid 1 2023-01-25T05:22:26.596 INFO:tasks.rados.rados.0.smithi016.stdout:19: writing smithi01698494-19 from 1903617 to 2302526 tid 2 2023-01-25T05:22:26.596 INFO:tasks.rados.rados.0.smithi016.stdout:5: done (15 left) 2023-01-25T05:22:26.597 INFO:tasks.rados.rados.0.smithi016.stdout:8: done (14 left) 2023-01-25T05:22:26.597 INFO:tasks.rados.rados.0.smithi016.stdout:20: write initial oid 20 2023-01-25T05:22:26.597 INFO:tasks.rados.rados.0.smithi016.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2023-01-25T05:22:26.603 INFO:tasks.rados.rados.0.smithi016.stdout:20: writing smithi01698494-20 from 719333 to 1181770 tid 1 2023-01-25T05:22:26.611 INFO:tasks.rados.rados.0.smithi016.stdout:20: writing smithi01698494-20 from 1962099 to 2319333 tid 2 2023-01-25T05:22:26.611 INFO:tasks.rados.rados.0.smithi016.stdout:21: write initial oid 21 2023-01-25T05:22:26.611 INFO:tasks.rados.rados.0.smithi016.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2023-01-25T05:22:26.618 INFO:tasks.rados.rados.0.smithi016.stdout:21: writing smithi01698494-21 from 736140 to 1273826 tid 1 2023-01-25T05:22:26.626 INFO:tasks.rados.rados.0.smithi016.stdout:21: writing smithi01698494-21 from 2020581 to 2336140 tid 2 2023-01-25T05:22:26.626 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.626 INFO:tasks.rados.rados.0.smithi016.stdout:9: finishing write tid 1 to smithi01698494-9 2023-01-25T05:22:26.626 INFO:tasks.rados.rados.0.smithi016.stdout:9: finishing write tid 2 to smithi01698494-9 2023-01-25T05:22:26.626 INFO:tasks.rados.rados.0.smithi016.stdout:9: finishing write tid 3 to smithi01698494-9 2023-01-25T05:22:26.627 INFO:tasks.rados.rados.0.smithi016.stdout:9: finishing write tid 4 to smithi01698494-9 2023-01-25T05:22:26.627 INFO:tasks.rados.rados.0.smithi016.stdout:9: oid 9 updating version 0 to 6 2023-01-25T05:22:26.627 INFO:tasks.rados.rados.0.smithi016.stdout:9: oid 9 version 6 is already newer than 5 2023-01-25T05:22:26.627 INFO:tasks.rados.rados.0.smithi016.stdout:9: oid 9 version 6 is already newer than 2 2023-01-25T05:22:26.627 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2023-01-25T05:22:26.628 INFO:tasks.rados.rados.0.smithi016.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2023-01-25T05:22:26.628 INFO:tasks.rados.rados.0.smithi016.stdout:10: finishing write tid 1 to smithi01698494-10 2023-01-25T05:22:26.628 INFO:tasks.rados.rados.0.smithi016.stdout:10: finishing write tid 2 to smithi01698494-10 2023-01-25T05:22:26.628 INFO:tasks.rados.rados.0.smithi016.stdout:10: finishing write tid 3 to smithi01698494-10 2023-01-25T05:22:26.628 INFO:tasks.rados.rados.0.smithi016.stdout:10: finishing write tid 4 to smithi01698494-10 2023-01-25T05:22:26.629 INFO:tasks.rados.rados.0.smithi016.stdout:10: oid 10 updating version 0 to 3 2023-01-25T05:22:26.629 INFO:tasks.rados.rados.0.smithi016.stdout:10: oid 10 updating version 3 to 8 2023-01-25T05:22:26.629 INFO:tasks.rados.rados.0.smithi016.stdout:10: oid 10 updating version 8 to 9 2023-01-25T05:22:26.629 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2023-01-25T05:22:26.629 INFO:tasks.rados.rados.0.smithi016.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2023-01-25T05:22:26.630 INFO:tasks.rados.rados.0.smithi016.stdout:11: finishing write tid 1 to smithi01698494-11 2023-01-25T05:22:26.630 INFO:tasks.rados.rados.0.smithi016.stdout:11: finishing write tid 2 to smithi01698494-11 2023-01-25T05:22:26.630 INFO:tasks.rados.rados.0.smithi016.stdout:11: finishing write tid 3 to smithi01698494-11 2023-01-25T05:22:26.630 INFO:tasks.rados.rados.0.smithi016.stdout:11: finishing write tid 4 to smithi01698494-11 2023-01-25T05:22:26.630 INFO:tasks.rados.rados.0.smithi016.stdout:11: oid 11 updating version 0 to 8 2023-01-25T05:22:26.631 INFO:tasks.rados.rados.0.smithi016.stdout:11: oid 11 version 8 is already newer than 3 2023-01-25T05:22:26.631 INFO:tasks.rados.rados.0.smithi016.stdout:11: oid 11 updating version 8 to 9 2023-01-25T05:22:26.631 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 11 v 9 (ObjNum 10 snap 0 seq_num 10) dirty exists 2023-01-25T05:22:26.631 INFO:tasks.rados.rados.0.smithi016.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2023-01-25T05:22:26.631 INFO:tasks.rados.rados.0.smithi016.stdout:12: finishing write tid 1 to smithi01698494-12 2023-01-25T05:22:26.632 INFO:tasks.rados.rados.0.smithi016.stdout:12: finishing write tid 2 to smithi01698494-12 2023-01-25T05:22:26.632 INFO:tasks.rados.rados.0.smithi016.stdout:12: finishing write tid 3 to smithi01698494-12 2023-01-25T05:22:26.632 INFO:tasks.rados.rados.0.smithi016.stdout:9: done (15 left) 2023-01-25T05:22:26.632 INFO:tasks.rados.rados.0.smithi016.stdout:10: done (14 left) 2023-01-25T05:22:26.632 INFO:tasks.rados.rados.0.smithi016.stdout:11: done (13 left) 2023-01-25T05:22:26.633 INFO:tasks.rados.rados.0.smithi016.stdout:22: write initial oid 22 2023-01-25T05:22:26.633 INFO:tasks.rados.rados.0.smithi016.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2023-01-25T05:22:26.633 INFO:tasks.rados.rados.0.smithi016.stdout:22: writing smithi01698494-22 from 752947 to 1365882 tid 1 2023-01-25T05:22:26.638 INFO:tasks.rados.rados.0.smithi016.stdout:22: writing smithi01698494-22 from 1962710 to 2352947 tid 2 2023-01-25T05:22:26.638 INFO:tasks.rados.rados.0.smithi016.stdout:23: write initial oid 23 2023-01-25T05:22:26.638 INFO:tasks.rados.rados.0.smithi016.stdout:12: finishing write tid 4 to smithi01698494-12 2023-01-25T05:22:26.638 INFO:tasks.rados.rados.0.smithi016.stdout:12: oid 12 updating version 0 to 9 2023-01-25T05:22:26.639 INFO:tasks.rados.rados.0.smithi016.stdout:12: oid 12 version 9 is already newer than 3 2023-01-25T05:22:26.639 INFO:tasks.rados.rados.0.smithi016.stdout:12: oid 12 version 9 is already newer than 8 2023-01-25T05:22:26.639 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 12 v 9 (ObjNum 11 snap 0 seq_num 11) dirty exists 2023-01-25T05:22:26.639 INFO:tasks.rados.rados.0.smithi016.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2023-01-25T05:22:26.639 INFO:tasks.rados.rados.0.smithi016.stdout:13: finishing write tid 1 to smithi01698494-13 2023-01-25T05:22:26.640 INFO:tasks.rados.rados.0.smithi016.stdout:13: finishing write tid 2 to smithi01698494-13 2023-01-25T05:22:26.640 INFO:tasks.rados.rados.0.smithi016.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2023-01-25T05:22:26.645 INFO:tasks.rados.rados.0.smithi016.stdout:23: writing smithi01698494-23 from 769754 to 1457938 tid 1 2023-01-25T05:22:26.651 INFO:tasks.rados.rados.0.smithi016.stdout:23: writing smithi01698494-23 from 2021192 to 2369754 tid 2 2023-01-25T05:22:26.651 INFO:tasks.rados.rados.0.smithi016.stdout:12: done (14 left) 2023-01-25T05:22:26.651 INFO:tasks.rados.rados.0.smithi016.stdout:24: write initial oid 24 2023-01-25T05:22:26.652 INFO:tasks.rados.rados.0.smithi016.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2023-01-25T05:22:26.657 INFO:tasks.rados.rados.0.smithi016.stdout:24: writing smithi01698494-24 from 786561 to 1266347 tid 1 2023-01-25T05:22:26.663 INFO:tasks.rados.rados.0.smithi016.stdout:24: writing smithi01698494-24 from 1796027 to 2263691 tid 2 2023-01-25T05:22:26.664 INFO:tasks.rados.rados.0.smithi016.stdout:25: write initial oid 25 2023-01-25T05:22:26.664 INFO:tasks.rados.rados.0.smithi016.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2023-01-25T05:22:26.668 INFO:tasks.rados.rados.0.smithi016.stdout:25: writing smithi01698494-25 from 403368 to 958403 tid 1 2023-01-25T05:22:26.675 INFO:tasks.rados.rados.0.smithi016.stdout:25: writing smithi01698494-25 from 1454509 to 2182184 tid 2 2023-01-25T05:22:26.676 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.676 INFO:tasks.rados.rados.0.smithi016.stdout:13: finishing write tid 3 to smithi01698494-13 2023-01-25T05:22:26.676 INFO:tasks.rados.rados.0.smithi016.stdout:13: finishing write tid 4 to smithi01698494-13 2023-01-25T05:22:26.676 INFO:tasks.rados.rados.0.smithi016.stdout:13: oid 13 updating version 0 to 4 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:13: oid 13 updating version 4 to 11 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:13: oid 13 updating version 11 to 12 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 13 v 12 (ObjNum 12 snap 0 seq_num 12) dirty exists 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:14: finishing write tid 1 to smithi01698494-14 2023-01-25T05:22:26.677 INFO:tasks.rados.rados.0.smithi016.stdout:13: done (15 left) 2023-01-25T05:22:26.678 INFO:tasks.rados.rados.0.smithi016.stdout:26: write initial oid 26 2023-01-25T05:22:26.678 INFO:tasks.rados.rados.0.smithi016.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2023-01-25T05:22:26.681 INFO:tasks.rados.rados.0.smithi016.stdout:26: writing smithi01698494-26 from 420175 to 1050459 tid 1 2023-01-25T05:22:26.689 INFO:tasks.rados.rados.0.smithi016.stdout:26: writing smithi01698494-26 from 1796638 to 2267971 tid 2 2023-01-25T05:22:26.689 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.689 INFO:tasks.rados.rados.0.smithi016.stdout:14: finishing write tid 2 to smithi01698494-14 2023-01-25T05:22:26.689 INFO:tasks.rados.rados.0.smithi016.stdout:14: finishing write tid 3 to smithi01698494-14 2023-01-25T05:22:26.689 INFO:tasks.rados.rados.0.smithi016.stdout:14: finishing write tid 4 to smithi01698494-14 2023-01-25T05:22:26.690 INFO:tasks.rados.rados.0.smithi016.stdout:14: oid 14 updating version 0 to 5 2023-01-25T05:22:26.690 INFO:tasks.rados.rados.0.smithi016.stdout:14: oid 14 updating version 5 to 15 2023-01-25T05:22:26.690 INFO:tasks.rados.rados.0.smithi016.stdout:14: oid 14 version 15 is already newer than 14 2023-01-25T05:22:26.690 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 14 v 15 (ObjNum 13 snap 0 seq_num 13) dirty exists 2023-01-25T05:22:26.690 INFO:tasks.rados.rados.0.smithi016.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2023-01-25T05:22:26.691 INFO:tasks.rados.rados.0.smithi016.stdout:15: finishing write tid 1 to smithi01698494-15 2023-01-25T05:22:26.691 INFO:tasks.rados.rados.0.smithi016.stdout:15: finishing write tid 2 to smithi01698494-15 2023-01-25T05:22:26.691 INFO:tasks.rados.rados.0.smithi016.stdout:15: finishing write tid 3 to smithi01698494-15 2023-01-25T05:22:26.691 INFO:tasks.rados.rados.0.smithi016.stdout:15: finishing write tid 4 to smithi01698494-15 2023-01-25T05:22:26.691 INFO:tasks.rados.rados.0.smithi016.stdout:15: oid 15 updating version 0 to 4 2023-01-25T05:22:26.692 INFO:tasks.rados.rados.0.smithi016.stdout:15: oid 15 updating version 4 to 11 2023-01-25T05:22:26.692 INFO:tasks.rados.rados.0.smithi016.stdout:15: oid 15 updating version 11 to 12 2023-01-25T05:22:26.692 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 15 v 12 (ObjNum 14 snap 0 seq_num 14) dirty exists 2023-01-25T05:22:26.692 INFO:tasks.rados.rados.0.smithi016.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2023-01-25T05:22:26.692 INFO:tasks.rados.rados.0.smithi016.stdout:14: done (15 left) 2023-01-25T05:22:26.693 INFO:tasks.rados.rados.0.smithi016.stdout:15: done (14 left) 2023-01-25T05:22:26.693 INFO:tasks.rados.rados.0.smithi016.stdout:27: write initial oid 27 2023-01-25T05:22:26.693 INFO:tasks.rados.rados.0.smithi016.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2023-01-25T05:22:26.694 INFO:tasks.rados.rados.0.smithi016.stdout:27: writing smithi01698494-27 from 436982 to 1142515 tid 1 2023-01-25T05:22:26.701 INFO:tasks.rados.rados.0.smithi016.stdout:27: writing smithi01698494-27 from 1855120 to 2436982 tid 2 2023-01-25T05:22:26.702 INFO:tasks.rados.rados.0.smithi016.stdout:28: write initial oid 28 2023-01-25T05:22:26.702 INFO:tasks.rados.rados.0.smithi016.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2023-01-25T05:22:26.708 INFO:tasks.rados.rados.0.smithi016.stdout:28: writing smithi01698494-28 from 453789 to 1234571 tid 1 2023-01-25T05:22:26.715 INFO:tasks.rados.rados.0.smithi016.stdout:28: writing smithi01698494-28 from 1913602 to 2388604 tid 2 2023-01-25T05:22:26.715 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.715 INFO:tasks.rados.rados.0.smithi016.stdout:16: finishing write tid 1 to smithi01698494-16 2023-01-25T05:22:26.716 INFO:tasks.rados.rados.0.smithi016.stdout:16: finishing write tid 2 to smithi01698494-16 2023-01-25T05:22:26.716 INFO:tasks.rados.rados.0.smithi016.stdout:16: finishing write tid 3 to smithi01698494-16 2023-01-25T05:22:26.716 INFO:tasks.rados.rados.0.smithi016.stdout:16: finishing write tid 4 to smithi01698494-16 2023-01-25T05:22:26.716 INFO:tasks.rados.rados.0.smithi016.stdout:16: oid 16 updating version 0 to 11 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:16: oid 16 version 11 is already newer than 4 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:16: oid 16 updating version 11 to 12 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 16 v 12 (ObjNum 15 snap 0 seq_num 15) dirty exists 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:17: finishing write tid 1 to smithi01698494-17 2023-01-25T05:22:26.717 INFO:tasks.rados.rados.0.smithi016.stdout:16: done (15 left) 2023-01-25T05:22:26.718 INFO:tasks.rados.rados.0.smithi016.stdout:29: write initial oid 29 2023-01-25T05:22:26.718 INFO:tasks.rados.rados.0.smithi016.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2023-01-25T05:22:26.720 INFO:tasks.rados.rados.0.smithi016.stdout:29: writing smithi01698494-29 from 470596 to 926627 tid 1 2023-01-25T05:22:26.727 INFO:tasks.rados.rados.0.smithi016.stdout:29: writing smithi01698494-29 from 1572084 to 2190744 tid 2 2023-01-25T05:22:26.727 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.728 INFO:tasks.rados.rados.0.smithi016.stdout:17: finishing write tid 2 to smithi01698494-17 2023-01-25T05:22:26.728 INFO:tasks.rados.rados.0.smithi016.stdout:17: finishing write tid 3 to smithi01698494-17 2023-01-25T05:22:26.728 INFO:tasks.rados.rados.0.smithi016.stdout:17: finishing write tid 4 to smithi01698494-17 2023-01-25T05:22:26.728 INFO:tasks.rados.rados.0.smithi016.stdout:17: oid 17 updating version 0 to 15 2023-01-25T05:22:26.728 INFO:tasks.rados.rados.0.smithi016.stdout:17: oid 17 version 15 is already newer than 14 2023-01-25T05:22:26.729 INFO:tasks.rados.rados.0.smithi016.stdout:17: oid 17 version 15 is already newer than 5 2023-01-25T05:22:26.729 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 17 v 15 (ObjNum 16 snap 0 seq_num 16) dirty exists 2023-01-25T05:22:26.729 INFO:tasks.rados.rados.0.smithi016.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2023-01-25T05:22:26.729 INFO:tasks.rados.rados.0.smithi016.stdout:18: finishing write tid 1 to smithi01698494-18 2023-01-25T05:22:26.729 INFO:tasks.rados.rados.0.smithi016.stdout:17: done (15 left) 2023-01-25T05:22:26.730 INFO:tasks.rados.rados.0.smithi016.stdout:30: write initial oid 30 2023-01-25T05:22:26.730 INFO:tasks.rados.rados.0.smithi016.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2023-01-25T05:22:26.732 INFO:tasks.rados.rados.0.smithi016.stdout:30: writing smithi01698494-30 from 487403 to 1018683 tid 1 2023-01-25T05:22:26.738 INFO:tasks.rados.rados.0.smithi016.stdout:30: writing smithi01698494-30 from 1514213 to 1992884 tid 2 2023-01-25T05:22:26.738 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.738 INFO:tasks.rados.rados.0.smithi016.stdout:18: finishing write tid 2 to smithi01698494-18 2023-01-25T05:22:26.738 INFO:tasks.rados.rados.0.smithi016.stdout:18: finishing write tid 3 to smithi01698494-18 2023-01-25T05:22:26.739 INFO:tasks.rados.rados.0.smithi016.stdout:18: finishing write tid 4 to smithi01698494-18 2023-01-25T05:22:26.739 INFO:tasks.rados.rados.0.smithi016.stdout:18: oid 18 updating version 0 to 18 2023-01-25T05:22:26.739 INFO:tasks.rados.rados.0.smithi016.stdout:18: oid 18 version 18 is already newer than 17 2023-01-25T05:22:26.739 INFO:tasks.rados.rados.0.smithi016.stdout:18: oid 18 version 18 is already newer than 6 2023-01-25T05:22:26.739 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 18 v 18 (ObjNum 17 snap 0 seq_num 17) dirty exists 2023-01-25T05:22:26.740 INFO:tasks.rados.rados.0.smithi016.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2023-01-25T05:22:26.740 INFO:tasks.rados.rados.0.smithi016.stdout:19: finishing write tid 1 to smithi01698494-19 2023-01-25T05:22:26.740 INFO:tasks.rados.rados.0.smithi016.stdout:18: done (15 left) 2023-01-25T05:22:26.740 INFO:tasks.rados.rados.0.smithi016.stdout:31: write initial oid 31 2023-01-25T05:22:26.740 INFO:tasks.rados.rados.0.smithi016.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2023-01-25T05:22:26.744 INFO:tasks.rados.rados.0.smithi016.stdout:31: writing smithi01698494-31 from 504210 to 1110739 tid 1 2023-01-25T05:22:26.752 INFO:tasks.rados.rados.0.smithi016.stdout:31: writing smithi01698494-31 from 1572695 to 2195024 tid 2 2023-01-25T05:22:26.752 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.753 INFO:tasks.rados.rados.0.smithi016.stdout:19: finishing write tid 2 to smithi01698494-19 2023-01-25T05:22:26.753 INFO:tasks.rados.rados.0.smithi016.stdout:19: finishing write tid 3 to smithi01698494-19 2023-01-25T05:22:26.753 INFO:tasks.rados.rados.0.smithi016.stdout:19: finishing write tid 4 to smithi01698494-19 2023-01-25T05:22:26.753 INFO:tasks.rados.rados.0.smithi016.stdout:19: oid 19 updating version 0 to 21 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:19: oid 19 version 21 is already newer than 20 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:19: oid 19 version 21 is already newer than 7 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 19 v 21 (ObjNum 18 snap 0 seq_num 18) dirty exists 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:3: finishing write tid 1 to smithi01698494-3 2023-01-25T05:22:26.754 INFO:tasks.rados.rados.0.smithi016.stdout:20: finishing write tid 1 to smithi01698494-20 2023-01-25T05:22:26.755 INFO:tasks.rados.rados.0.smithi016.stdout:6: finishing write tid 1 to smithi01698494-6 2023-01-25T05:22:26.755 INFO:tasks.rados.rados.0.smithi016.stdout:20: finishing write tid 2 to smithi01698494-20 2023-01-25T05:22:26.755 INFO:tasks.rados.rados.0.smithi016.stdout:20: finishing write tid 3 to smithi01698494-20 2023-01-25T05:22:26.755 INFO:tasks.rados.rados.0.smithi016.stdout:20: finishing write tid 4 to smithi01698494-20 2023-01-25T05:22:26.755 INFO:tasks.rados.rados.0.smithi016.stdout:20: oid 20 updating version 0 to 17 2023-01-25T05:22:26.756 INFO:tasks.rados.rados.0.smithi016.stdout:20: oid 20 version 17 is already newer than 6 2023-01-25T05:22:26.756 INFO:tasks.rados.rados.0.smithi016.stdout:20: oid 20 updating version 17 to 18 2023-01-25T05:22:26.756 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 20 v 18 (ObjNum 19 snap 0 seq_num 19) dirty exists 2023-01-25T05:22:26.756 INFO:tasks.rados.rados.0.smithi016.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2023-01-25T05:22:26.756 INFO:tasks.rados.rados.0.smithi016.stdout:7: finishing write tid 1 to smithi01698494-7 2023-01-25T05:22:26.757 INFO:tasks.rados.rados.0.smithi016.stdout:3: finishing write tid 2 to smithi01698494-3 2023-01-25T05:22:26.757 INFO:tasks.rados.rados.0.smithi016.stdout:3: finishing write tid 3 to smithi01698494-3 2023-01-25T05:22:26.757 INFO:tasks.rados.rados.0.smithi016.stdout:3: finishing write tid 4 to smithi01698494-3 2023-01-25T05:22:26.757 INFO:tasks.rados.rados.0.smithi016.stdout:3: oid 3 updating version 0 to 2 2023-01-25T05:22:26.757 INFO:tasks.rados.rados.0.smithi016.stdout:3: oid 3 updating version 2 to 3 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:3: oid 3 version 3 is already newer than 1 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:6: finishing write tid 2 to smithi01698494-6 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:6: finishing write tid 3 to smithi01698494-6 2023-01-25T05:22:26.758 INFO:tasks.rados.rados.0.smithi016.stdout:6: finishing write tid 4 to smithi01698494-6 2023-01-25T05:22:26.759 INFO:tasks.rados.rados.0.smithi016.stdout:6: oid 6 updating version 0 to 6 2023-01-25T05:22:26.759 INFO:tasks.rados.rados.0.smithi016.stdout:6: oid 6 version 6 is already newer than 5 2023-01-25T05:22:26.759 INFO:tasks.rados.rados.0.smithi016.stdout:6: oid 6 version 6 is already newer than 2 2023-01-25T05:22:26.759 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2023-01-25T05:22:26.759 INFO:tasks.rados.rados.0.smithi016.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2023-01-25T05:22:26.760 INFO:tasks.rados.rados.0.smithi016.stdout:7: finishing write tid 2 to smithi01698494-7 2023-01-25T05:22:26.760 INFO:tasks.rados.rados.0.smithi016.stdout:7: finishing write tid 3 to smithi01698494-7 2023-01-25T05:22:26.760 INFO:tasks.rados.rados.0.smithi016.stdout:7: finishing write tid 4 to smithi01698494-7 2023-01-25T05:22:26.760 INFO:tasks.rados.rados.0.smithi016.stdout:7: oid 7 updating version 0 to 3 2023-01-25T05:22:26.760 INFO:tasks.rados.rados.0.smithi016.stdout:7: oid 7 updating version 3 to 8 2023-01-25T05:22:26.761 INFO:tasks.rados.rados.0.smithi016.stdout:7: oid 7 updating version 8 to 9 2023-01-25T05:22:26.761 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 7 v 9 (ObjNum 6 snap 0 seq_num 6) dirty exists 2023-01-25T05:22:26.761 INFO:tasks.rados.rados.0.smithi016.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2023-01-25T05:22:26.761 INFO:tasks.rados.rados.0.smithi016.stdout:21: finishing write tid 1 to smithi01698494-21 2023-01-25T05:22:26.761 INFO:tasks.rados.rados.0.smithi016.stdout:21: finishing write tid 2 to smithi01698494-21 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:21: finishing write tid 3 to smithi01698494-21 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:21: finishing write tid 4 to smithi01698494-21 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:21: oid 21 updating version 0 to 15 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:21: oid 21 version 15 is already newer than 14 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:21: oid 21 version 15 is already newer than 5 2023-01-25T05:22:26.762 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 21 v 15 (ObjNum 20 snap 0 seq_num 20) dirty exists 2023-01-25T05:22:26.763 INFO:tasks.rados.rados.0.smithi016.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2023-01-25T05:22:26.763 INFO:tasks.rados.rados.0.smithi016.stdout:22: finishing write tid 1 to smithi01698494-22 2023-01-25T05:22:26.763 INFO:tasks.rados.rados.0.smithi016.stdout:22: finishing write tid 2 to smithi01698494-22 2023-01-25T05:22:26.763 INFO:tasks.rados.rados.0.smithi016.stdout:22: finishing write tid 3 to smithi01698494-22 2023-01-25T05:22:26.763 INFO:tasks.rados.rados.0.smithi016.stdout:22: finishing write tid 4 to smithi01698494-22 2023-01-25T05:22:26.764 INFO:tasks.rados.rados.0.smithi016.stdout:22: oid 22 updating version 0 to 8 2023-01-25T05:22:26.764 INFO:tasks.rados.rados.0.smithi016.stdout:22: oid 22 updating version 8 to 24 2023-01-25T05:22:26.764 INFO:tasks.rados.rados.0.smithi016.stdout:22: oid 22 version 24 is already newer than 23 2023-01-25T05:22:26.764 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 22 v 24 (ObjNum 21 snap 0 seq_num 21) dirty exists 2023-01-25T05:22:26.764 INFO:tasks.rados.rados.0.smithi016.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:23: finishing write tid 1 to smithi01698494-23 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:23: finishing write tid 2 to smithi01698494-23 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:3: done (15 left) 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:6: done (14 left) 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:7: done (13 left) 2023-01-25T05:22:26.765 INFO:tasks.rados.rados.0.smithi016.stdout:19: done (12 left) 2023-01-25T05:22:26.766 INFO:tasks.rados.rados.0.smithi016.stdout:20: done (11 left) 2023-01-25T05:22:26.766 INFO:tasks.rados.rados.0.smithi016.stdout:21: done (10 left) 2023-01-25T05:22:26.766 INFO:tasks.rados.rados.0.smithi016.stdout:22: done (9 left) 2023-01-25T05:22:26.766 INFO:tasks.rados.rados.0.smithi016.stdout:32: write initial oid 32 2023-01-25T05:22:26.766 INFO:tasks.rados.rados.0.smithi016.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2023-01-25T05:22:26.767 INFO:tasks.rados.rados.0.smithi016.stdout:32: writing smithi01698494-32 from 521017 to 1319148 tid 1 2023-01-25T05:22:26.767 INFO:tasks.rados.rados.0.smithi016.stdout:32: writing smithi01698494-32 from 1747530 to 2229870 tid 2 2023-01-25T05:22:26.768 INFO:tasks.rados.rados.0.smithi016.stdout:33: write initial oid 33 2023-01-25T05:22:26.768 INFO:tasks.rados.rados.0.smithi016.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2023-01-25T05:22:26.768 INFO:tasks.rados.rados.0.smithi016.stdout:33: writing smithi01698494-33 from 537824 to 1011204 tid 1 2023-01-25T05:22:26.776 INFO:tasks.rados.rados.0.smithi016.stdout:33: writing smithi01698494-33 from 1806012 to 2432010 tid 2 2023-01-25T05:22:26.776 INFO:tasks.rados.rados.0.smithi016.stdout:34: write initial oid 34 2023-01-25T05:22:26.776 INFO:tasks.rados.rados.0.smithi016.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2023-01-25T05:22:26.782 INFO:tasks.rados.rados.0.smithi016.stdout:34: writing smithi01698494-34 from 554631 to 1103260 tid 1 2023-01-25T05:22:26.788 INFO:tasks.rados.rados.0.smithi016.stdout:34: writing smithi01698494-34 from 1748141 to 2234150 tid 2 2023-01-25T05:22:26.788 INFO:tasks.rados.rados.0.smithi016.stdout:35: write initial oid 35 2023-01-25T05:22:26.789 INFO:tasks.rados.rados.0.smithi016.stdout:23: finishing write tid 3 to smithi01698494-23 2023-01-25T05:22:26.789 INFO:tasks.rados.rados.0.smithi016.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2023-01-25T05:22:26.794 INFO:tasks.rados.rados.0.smithi016.stdout:35: writing smithi01698494-35 from 571438 to 1195316 tid 1 2023-01-25T05:22:26.802 INFO:tasks.rados.rados.0.smithi016.stdout:35: writing smithi01698494-35 from 1806623 to 2436290 tid 2 2023-01-25T05:22:26.802 INFO:tasks.rados.rados.0.smithi016.stdout:36: write initial oid 36 2023-01-25T05:22:26.802 INFO:tasks.rados.rados.0.smithi016.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2023-01-25T05:22:26.808 INFO:tasks.rados.rados.0.smithi016.stdout:36: writing smithi01698494-36 from 588245 to 1287372 tid 1 2023-01-25T05:22:26.815 INFO:tasks.rados.rados.0.smithi016.stdout:36: writing smithi01698494-36 from 1865105 to 2354783 tid 2 2023-01-25T05:22:26.815 INFO:tasks.rados.rados.0.smithi016.stdout:37: write initial oid 37 2023-01-25T05:22:26.815 INFO:tasks.rados.rados.0.smithi016.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2023-01-25T05:22:26.822 INFO:tasks.rados.rados.0.smithi016.stdout:37: writing smithi01698494-37 from 605052 to 1379428 tid 1 2023-01-25T05:22:26.829 INFO:tasks.rados.rados.0.smithi016.stdout:37: writing smithi01698494-37 from 1923587 to 2556923 tid 2 2023-01-25T05:22:26.830 INFO:tasks.rados.rados.0.smithi016.stdout:38: write initial oid 38 2023-01-25T05:22:26.830 INFO:tasks.rados.rados.0.smithi016.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2023-01-25T05:22:26.835 INFO:tasks.rados.rados.0.smithi016.stdout:38: writing smithi01698494-38 from 621859 to 1071484 tid 1 2023-01-25T05:22:26.844 INFO:tasks.rados.rados.0.smithi016.stdout:38: writing smithi01698494-38 from 1865716 to 2621859 tid 2 2023-01-25T05:22:26.844 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.844 INFO:tasks.rados.rados.0.smithi016.stdout:23: finishing write tid 4 to smithi01698494-23 2023-01-25T05:22:26.845 INFO:tasks.rados.rados.0.smithi016.stdout:23: oid 23 updating version 0 to 26 2023-01-25T05:22:26.845 INFO:tasks.rados.rados.0.smithi016.stdout:23: oid 23 version 26 is already newer than 9 2023-01-25T05:22:26.845 INFO:tasks.rados.rados.0.smithi016.stdout:23: oid 23 updating version 26 to 27 2023-01-25T05:22:26.845 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 23 v 27 (ObjNum 22 snap 0 seq_num 22) dirty exists 2023-01-25T05:22:26.845 INFO:tasks.rados.rados.0.smithi016.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2023-01-25T05:22:26.846 INFO:tasks.rados.rados.0.smithi016.stdout:23: done (15 left) 2023-01-25T05:22:26.846 INFO:tasks.rados.rados.0.smithi016.stdout:39: write initial oid 39 2023-01-25T05:22:26.846 INFO:tasks.rados.rados.0.smithi016.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2023-01-25T05:22:26.850 INFO:tasks.rados.rados.0.smithi016.stdout:39: writing smithi01698494-39 from 638666 to 1163540 tid 1 2023-01-25T05:22:26.858 INFO:tasks.rados.rados.0.smithi016.stdout:39: writing smithi01698494-39 from 1924198 to 2561203 tid 2 2023-01-25T05:22:26.858 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.858 INFO:tasks.rados.rados.0.smithi016.stdout:24: finishing write tid 1 to smithi01698494-24 2023-01-25T05:22:26.859 INFO:tasks.rados.rados.0.smithi016.stdout:24: finishing write tid 2 to smithi01698494-24 2023-01-25T05:22:26.859 INFO:tasks.rados.rados.0.smithi016.stdout:24: finishing write tid 3 to smithi01698494-24 2023-01-25T05:22:26.859 INFO:tasks.rados.rados.0.smithi016.stdout:24: finishing write tid 4 to smithi01698494-24 2023-01-25T05:22:26.859 INFO:tasks.rados.rados.0.smithi016.stdout:24: oid 24 updating version 0 to 20 2023-01-25T05:22:26.859 INFO:tasks.rados.rados.0.smithi016.stdout:24: oid 24 version 20 is already newer than 7 2023-01-25T05:22:26.860 INFO:tasks.rados.rados.0.smithi016.stdout:24: oid 24 updating version 20 to 21 2023-01-25T05:22:26.860 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 24 v 21 (ObjNum 23 snap 0 seq_num 23) dirty exists 2023-01-25T05:22:26.860 INFO:tasks.rados.rados.0.smithi016.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2023-01-25T05:22:26.860 INFO:tasks.rados.rados.0.smithi016.stdout:25: finishing write tid 1 to smithi01698494-25 2023-01-25T05:22:26.860 INFO:tasks.rados.rados.0.smithi016.stdout:25: finishing write tid 2 to smithi01698494-25 2023-01-25T05:22:26.861 INFO:tasks.rados.rados.0.smithi016.stdout:24: done (15 left) 2023-01-25T05:22:26.861 INFO:tasks.rados.rados.0.smithi016.stdout:40: write initial oid 40 2023-01-25T05:22:26.861 INFO:tasks.rados.rados.0.smithi016.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2023-01-25T05:22:26.865 INFO:tasks.rados.rados.0.smithi016.stdout:40: writing smithi01698494-40 from 655473 to 1371949 tid 1 2023-01-25T05:22:26.873 INFO:tasks.rados.rados.0.smithi016.stdout:40: writing smithi01698494-40 from 2099033 to 2655473 tid 2 2023-01-25T05:22:26.873 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.873 INFO:tasks.rados.rados.0.smithi016.stdout:25: finishing write tid 3 to smithi01698494-25 2023-01-25T05:22:26.874 INFO:tasks.rados.rados.0.smithi016.stdout:25: finishing write tid 4 to smithi01698494-25 2023-01-25T05:22:26.874 INFO:tasks.rados.rados.0.smithi016.stdout:25: oid 25 updating version 0 to 12 2023-01-25T05:22:26.874 INFO:tasks.rados.rados.0.smithi016.stdout:25: oid 25 version 12 is already newer than 4 2023-01-25T05:22:26.874 INFO:tasks.rados.rados.0.smithi016.stdout:25: oid 25 version 12 is already newer than 11 2023-01-25T05:22:26.875 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 25 v 12 (ObjNum 24 snap 0 seq_num 24) dirty exists 2023-01-25T05:22:26.875 INFO:tasks.rados.rados.0.smithi016.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2023-01-25T05:22:26.875 INFO:tasks.rados.rados.0.smithi016.stdout:26: finishing write tid 1 to smithi01698494-26 2023-01-25T05:22:26.875 INFO:tasks.rados.rados.0.smithi016.stdout:26: finishing write tid 2 to smithi01698494-26 2023-01-25T05:22:26.875 INFO:tasks.rados.rados.0.smithi016.stdout:26: finishing write tid 3 to smithi01698494-26 2023-01-25T05:22:26.876 INFO:tasks.rados.rados.0.smithi016.stdout:26: finishing write tid 4 to smithi01698494-26 2023-01-25T05:22:26.876 INFO:tasks.rados.rados.0.smithi016.stdout:26: oid 26 updating version 0 to 8 2023-01-25T05:22:26.876 INFO:tasks.rados.rados.0.smithi016.stdout:26: oid 26 updating version 8 to 24 2023-01-25T05:22:26.876 INFO:tasks.rados.rados.0.smithi016.stdout:26: oid 26 version 24 is already newer than 23 2023-01-25T05:22:26.876 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 26 v 24 (ObjNum 25 snap 0 seq_num 25) dirty exists 2023-01-25T05:22:26.877 INFO:tasks.rados.rados.0.smithi016.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2023-01-25T05:22:26.877 INFO:tasks.rados.rados.0.smithi016.stdout:25: done (15 left) 2023-01-25T05:22:26.877 INFO:tasks.rados.rados.0.smithi016.stdout:26: done (14 left) 2023-01-25T05:22:26.877 INFO:tasks.rados.rados.0.smithi016.stdout:41: write initial oid 41 2023-01-25T05:22:26.877 INFO:tasks.rados.rados.0.smithi016.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2023-01-25T05:22:26.880 INFO:tasks.rados.rados.0.smithi016.stdout:41: writing smithi01698494-41 from 672280 to 1464005 tid 1 2023-01-25T05:22:26.887 INFO:tasks.rados.rados.0.smithi016.stdout:41: writing smithi01698494-41 from 2157515 to 2672280 tid 2 2023-01-25T05:22:26.887 INFO:tasks.rados.rados.0.smithi016.stdout:42: write initial oid 42 2023-01-25T05:22:26.888 INFO:tasks.rados.rados.0.smithi016.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2023-01-25T05:22:26.894 INFO:tasks.rados.rados.0.smithi016.stdout:42: writing smithi01698494-42 from 689087 to 1156061 tid 1 2023-01-25T05:22:26.901 INFO:tasks.rados.rados.0.smithi016.stdout:42: writing smithi01698494-42 from 1699644 to 2483976 tid 2 2023-01-25T05:22:26.902 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.902 INFO:tasks.rados.rados.0.smithi016.stdout:27: finishing write tid 1 to smithi01698494-27 2023-01-25T05:22:26.902 INFO:tasks.rados.rados.0.smithi016.stdout:27: finishing write tid 2 to smithi01698494-27 2023-01-25T05:22:26.902 INFO:tasks.rados.rados.0.smithi016.stdout:27: finishing write tid 3 to smithi01698494-27 2023-01-25T05:22:26.903 INFO:tasks.rados.rados.0.smithi016.stdout:27: finishing write tid 4 to smithi01698494-27 2023-01-25T05:22:26.903 INFO:tasks.rados.rados.0.smithi016.stdout:27: oid 27 updating version 0 to 18 2023-01-25T05:22:26.903 INFO:tasks.rados.rados.0.smithi016.stdout:27: oid 27 version 18 is already newer than 17 2023-01-25T05:22:26.903 INFO:tasks.rados.rados.0.smithi016.stdout:27: oid 27 version 18 is already newer than 6 2023-01-25T05:22:26.903 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 27 v 18 (ObjNum 26 snap 0 seq_num 26) dirty exists 2023-01-25T05:22:26.904 INFO:tasks.rados.rados.0.smithi016.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2023-01-25T05:22:26.904 INFO:tasks.rados.rados.0.smithi016.stdout:28: finishing write tid 1 to smithi01698494-28 2023-01-25T05:22:26.904 INFO:tasks.rados.rados.0.smithi016.stdout:27: done (15 left) 2023-01-25T05:22:26.904 INFO:tasks.rados.rados.0.smithi016.stdout:43: write initial oid 43 2023-01-25T05:22:26.904 INFO:tasks.rados.rados.0.smithi016.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2023-01-25T05:22:26.907 INFO:tasks.rados.rados.0.smithi016.stdout:43: writing smithi01698494-43 from 705894 to 1248117 tid 1 2023-01-25T05:22:26.915 INFO:tasks.rados.rados.0.smithi016.stdout:43: writing smithi01698494-43 from 1758126 to 2402469 tid 2 2023-01-25T05:22:26.915 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.915 INFO:tasks.rados.rados.0.smithi016.stdout:28: finishing write tid 2 to smithi01698494-28 2023-01-25T05:22:26.915 INFO:tasks.rados.rados.0.smithi016.stdout:28: finishing write tid 3 to smithi01698494-28 2023-01-25T05:22:26.915 INFO:tasks.rados.rados.0.smithi016.stdout:28: finishing write tid 4 to smithi01698494-28 2023-01-25T05:22:26.916 INFO:tasks.rados.rados.0.smithi016.stdout:28: oid 28 updating version 0 to 26 2023-01-25T05:22:26.916 INFO:tasks.rados.rados.0.smithi016.stdout:28: oid 28 version 26 is already newer than 9 2023-01-25T05:22:26.916 INFO:tasks.rados.rados.0.smithi016.stdout:28: oid 28 updating version 26 to 27 2023-01-25T05:22:26.916 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 28 v 27 (ObjNum 27 snap 0 seq_num 27) dirty exists 2023-01-25T05:22:26.917 INFO:tasks.rados.rados.0.smithi016.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2023-01-25T05:22:26.917 INFO:tasks.rados.rados.0.smithi016.stdout:28: done (15 left) 2023-01-25T05:22:26.917 INFO:tasks.rados.rados.0.smithi016.stdout:44: write initial oid 44 2023-01-25T05:22:26.917 INFO:tasks.rados.rados.0.smithi016.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2023-01-25T05:22:26.921 INFO:tasks.rados.rados.0.smithi016.stdout:44: writing smithi01698494-44 from 722701 to 1340173 tid 1 2023-01-25T05:22:26.930 INFO:tasks.rados.rados.0.smithi016.stdout:44: writing smithi01698494-44 from 1816608 to 2604609 tid 2 2023-01-25T05:22:26.930 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.930 INFO:tasks.rados.rados.0.smithi016.stdout:29: finishing write tid 1 to smithi01698494-29 2023-01-25T05:22:26.931 INFO:tasks.rados.rados.0.smithi016.stdout:29: finishing write tid 2 to smithi01698494-29 2023-01-25T05:22:26.931 INFO:tasks.rados.rados.0.smithi016.stdout:29: finishing write tid 3 to smithi01698494-29 2023-01-25T05:22:26.931 INFO:tasks.rados.rados.0.smithi016.stdout:29: finishing write tid 4 to smithi01698494-29 2023-01-25T05:22:26.931 INFO:tasks.rados.rados.0.smithi016.stdout:29: oid 29 updating version 0 to 14 2023-01-25T05:22:26.931 INFO:tasks.rados.rados.0.smithi016.stdout:29: oid 29 version 14 is already newer than 5 2023-01-25T05:22:26.932 INFO:tasks.rados.rados.0.smithi016.stdout:29: oid 29 updating version 14 to 15 2023-01-25T05:22:26.932 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 29 v 15 (ObjNum 28 snap 0 seq_num 28) dirty exists 2023-01-25T05:22:26.932 INFO:tasks.rados.rados.0.smithi016.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2023-01-25T05:22:26.932 INFO:tasks.rados.rados.0.smithi016.stdout:30: finishing write tid 1 to smithi01698494-30 2023-01-25T05:22:26.932 INFO:tasks.rados.rados.0.smithi016.stdout:29: done (15 left) 2023-01-25T05:22:26.933 INFO:tasks.rados.rados.0.smithi016.stdout:45: write initial oid 45 2023-01-25T05:22:26.933 INFO:tasks.rados.rados.0.smithi016.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2023-01-25T05:22:26.936 INFO:tasks.rados.rados.0.smithi016.stdout:45: writing smithi01698494-45 from 739508 to 1432229 tid 1 2023-01-25T05:22:26.946 INFO:tasks.rados.rados.0.smithi016.stdout:45: writing smithi01698494-45 from 1875090 to 2523102 tid 2 2023-01-25T05:22:26.946 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:26.946 INFO:tasks.rados.rados.0.smithi016.stdout:30: finishing write tid 2 to smithi01698494-30 2023-01-25T05:22:26.946 INFO:tasks.rados.rados.0.smithi016.stdout:30: finishing write tid 3 to smithi01698494-30 2023-01-25T05:22:26.947 INFO:tasks.rados.rados.0.smithi016.stdout:30: finishing write tid 4 to smithi01698494-30 2023-01-25T05:22:26.947 INFO:tasks.rados.rados.0.smithi016.stdout:30: oid 30 updating version 0 to 10 2023-01-25T05:22:26.947 INFO:tasks.rados.rados.0.smithi016.stdout:30: oid 30 updating version 10 to 11 2023-01-25T05:22:26.947 INFO:tasks.rados.rados.0.smithi016.stdout:30: oid 30 updating version 11 to 12 2023-01-25T05:22:26.948 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 30 v 12 (ObjNum 29 snap 0 seq_num 29) dirty exists 2023-01-25T05:22:26.948 INFO:tasks.rados.rados.0.smithi016.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2023-01-25T05:22:26.948 INFO:tasks.rados.rados.0.smithi016.stdout:31: finishing write tid 1 to smithi01698494-31 2023-01-25T05:22:26.948 INFO:tasks.rados.rados.0.smithi016.stdout:31: finishing write tid 2 to smithi01698494-31 2023-01-25T05:22:26.948 INFO:tasks.rados.rados.0.smithi016.stdout:31: finishing write tid 3 to smithi01698494-31 2023-01-25T05:22:26.949 INFO:tasks.rados.rados.0.smithi016.stdout:31: finishing write tid 4 to smithi01698494-31 2023-01-25T05:22:26.949 INFO:tasks.rados.rados.0.smithi016.stdout:31: oid 31 updating version 0 to 10 2023-01-25T05:22:26.949 INFO:tasks.rados.rados.0.smithi016.stdout:31: oid 31 updating version 10 to 29 2023-01-25T05:22:26.949 INFO:tasks.rados.rados.0.smithi016.stdout:31: oid 31 updating version 29 to 30 2023-01-25T05:22:26.949 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 31 v 30 (ObjNum 30 snap 0 seq_num 30) dirty exists 2023-01-25T05:22:26.950 INFO:tasks.rados.rados.0.smithi016.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2023-01-25T05:22:26.950 INFO:tasks.rados.rados.0.smithi016.stdout:32: finishing write tid 1 to smithi01698494-32 2023-01-25T05:22:26.950 INFO:tasks.rados.rados.0.smithi016.stdout:32: finishing write tid 2 to smithi01698494-32 2023-01-25T05:22:26.950 INFO:tasks.rados.rados.0.smithi016.stdout:32: finishing write tid 3 to smithi01698494-32 2023-01-25T05:22:26.950 INFO:tasks.rados.rados.0.smithi016.stdout:32: finishing write tid 4 to smithi01698494-32 2023-01-25T05:22:26.951 INFO:tasks.rados.rados.0.smithi016.stdout:32: oid 32 updating version 0 to 15 2023-01-25T05:22:26.951 INFO:tasks.rados.rados.0.smithi016.stdout:32: oid 32 version 15 is already newer than 14 2023-01-25T05:22:26.951 INFO:tasks.rados.rados.0.smithi016.stdout:32: oid 32 version 15 is already newer than 13 2023-01-25T05:22:26.951 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 32 v 15 (ObjNum 31 snap 0 seq_num 31) dirty exists 2023-01-25T05:22:26.951 INFO:tasks.rados.rados.0.smithi016.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2023-01-25T05:22:26.952 INFO:tasks.rados.rados.0.smithi016.stdout:30: done (15 left) 2023-01-25T05:22:26.952 INFO:tasks.rados.rados.0.smithi016.stdout:31: done (14 left) 2023-01-25T05:22:26.952 INFO:tasks.rados.rados.0.smithi016.stdout:32: done (13 left) 2023-01-25T05:22:26.952 INFO:tasks.rados.rados.0.smithi016.stdout:46: write initial oid 46 2023-01-25T05:22:26.952 INFO:tasks.rados.rados.0.smithi016.stdout:33: finishing write tid 1 to smithi01698494-33 2023-01-25T05:22:26.953 INFO:tasks.rados.rados.0.smithi016.stdout:33: finishing write tid 2 to smithi01698494-33 2023-01-25T05:22:26.953 INFO:tasks.rados.rados.0.smithi016.stdout:33: finishing write tid 3 to smithi01698494-33 2023-01-25T05:22:26.953 INFO:tasks.rados.rados.0.smithi016.stdout:33: finishing write tid 4 to smithi01698494-33 2023-01-25T05:22:26.953 INFO:tasks.rados.rados.0.smithi016.stdout:33: oid 33 updating version 0 to 18 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:33: oid 33 version 18 is already newer than 16 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:33: oid 33 version 18 is already newer than 17 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 33 v 18 (ObjNum 32 snap 0 seq_num 32) dirty exists 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:34: finishing write tid 1 to smithi01698494-34 2023-01-25T05:22:26.954 INFO:tasks.rados.rados.0.smithi016.stdout:34: finishing write tid 2 to smithi01698494-34 2023-01-25T05:22:26.955 INFO:tasks.rados.rados.0.smithi016.stdout:34: finishing write tid 3 to smithi01698494-34 2023-01-25T05:22:26.955 INFO:tasks.rados.rados.0.smithi016.stdout:34: finishing write tid 4 to smithi01698494-34 2023-01-25T05:22:26.955 INFO:tasks.rados.rados.0.smithi016.stdout:34: oid 34 updating version 0 to 20 2023-01-25T05:22:26.955 INFO:tasks.rados.rados.0.smithi016.stdout:34: oid 34 version 20 is already newer than 19 2023-01-25T05:22:26.955 INFO:tasks.rados.rados.0.smithi016.stdout:34: oid 34 updating version 20 to 21 2023-01-25T05:22:26.956 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 34 v 21 (ObjNum 33 snap 0 seq_num 33) dirty exists 2023-01-25T05:22:26.956 INFO:tasks.rados.rados.0.smithi016.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2023-01-25T05:22:26.956 INFO:tasks.rados.rados.0.smithi016.stdout:35: finishing write tid 1 to smithi01698494-35 2023-01-25T05:22:26.956 INFO:tasks.rados.rados.0.smithi016.stdout:35: finishing write tid 2 to smithi01698494-35 2023-01-25T05:22:26.957 INFO:tasks.rados.rados.0.smithi016.stdout:35: finishing write tid 3 to smithi01698494-35 2023-01-25T05:22:26.957 INFO:tasks.rados.rados.0.smithi016.stdout:35: finishing write tid 4 to smithi01698494-35 2023-01-25T05:22:26.957 INFO:tasks.rados.rados.0.smithi016.stdout:35: oid 35 updating version 0 to 7 2023-01-25T05:22:26.957 INFO:tasks.rados.rados.0.smithi016.stdout:35: oid 35 updating version 7 to 20 2023-01-25T05:22:26.957 INFO:tasks.rados.rados.0.smithi016.stdout:35: oid 35 updating version 20 to 21 2023-01-25T05:22:26.958 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 35 v 21 (ObjNum 34 snap 0 seq_num 34) dirty exists 2023-01-25T05:22:26.958 INFO:tasks.rados.rados.0.smithi016.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2023-01-25T05:22:26.958 INFO:tasks.rados.rados.0.smithi016.stdout:36: finishing write tid 1 to smithi01698494-36 2023-01-25T05:22:26.958 INFO:tasks.rados.rados.0.smithi016.stdout:36: finishing write tid 2 to smithi01698494-36 2023-01-25T05:22:26.958 INFO:tasks.rados.rados.0.smithi016.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2023-01-25T05:22:26.959 INFO:tasks.rados.rados.0.smithi016.stdout:46: writing smithi01698494-46 from 756315 to 1524285 tid 1 2023-01-25T05:22:26.959 INFO:tasks.rados.rados.0.smithi016.stdout:46: writing smithi01698494-46 from 1933572 to 2725242 tid 2 2023-01-25T05:22:26.960 INFO:tasks.rados.rados.0.smithi016.stdout:33: done (13 left) 2023-01-25T05:22:26.960 INFO:tasks.rados.rados.0.smithi016.stdout:34: done (12 left) 2023-01-25T05:22:26.960 INFO:tasks.rados.rados.0.smithi016.stdout:35: done (11 left) 2023-01-25T05:22:26.960 INFO:tasks.rados.rados.0.smithi016.stdout:47: write initial oid 47 2023-01-25T05:22:26.960 INFO:tasks.rados.rados.0.smithi016.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2023-01-25T05:22:26.965 INFO:tasks.rados.rados.0.smithi016.stdout:47: writing smithi01698494-47 from 773122 to 1332694 tid 1 2023-01-25T05:22:26.973 INFO:tasks.rados.rados.0.smithi016.stdout:47: writing smithi01698494-47 from 1992054 to 2643735 tid 2 2023-01-25T05:22:26.973 INFO:tasks.rados.rados.0.smithi016.stdout:48: write initial oid 48 2023-01-25T05:22:26.973 INFO:tasks.rados.rados.0.smithi016.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2023-01-25T05:22:26.981 INFO:tasks.rados.rados.0.smithi016.stdout:48: writing smithi01698494-48 from 789929 to 1424750 tid 1 2023-01-25T05:22:26.988 INFO:tasks.rados.rados.0.smithi016.stdout:48: writing smithi01698494-48 from 2050536 to 2789929 tid 2 2023-01-25T05:22:26.988 INFO:tasks.rados.rados.0.smithi016.stdout:49: write initial oid 49 2023-01-25T05:22:26.988 INFO:tasks.rados.rados.0.smithi016.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2023-01-25T05:22:26.994 INFO:tasks.rados.rados.0.smithi016.stdout:49: writing smithi01698494-49 from 406736 to 1116806 tid 1 2023-01-25T05:22:27.001 INFO:tasks.rados.rados.0.smithi016.stdout:49: writing smithi01698494-49 from 1709018 to 2364368 tid 2 2023-01-25T05:22:27.001 INFO:tasks.rados.rados.0.smithi016.stdout:50: write initial oid 50 2023-01-25T05:22:27.001 INFO:tasks.rados.rados.0.smithi016.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2023-01-25T05:22:27.008 INFO:tasks.rados.rados.0.smithi016.stdout:50: writing smithi01698494-50 from 423543 to 1208862 tid 1 2023-01-25T05:22:27.015 INFO:tasks.rados.rados.0.smithi016.stdout:50: writing smithi01698494-50 from 1767500 to 2566508 tid 2 2023-01-25T05:22:27.015 INFO:tasks.rados.rados.0.smithi016.stdout:51: write initial oid 51 2023-01-25T05:22:27.016 INFO:tasks.rados.rados.0.smithi016.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2023-01-25T05:22:27.020 INFO:tasks.rados.rados.0.smithi016.stdout:51: writing smithi01698494-51 from 440350 to 900918 tid 1 2023-01-25T05:22:27.026 INFO:tasks.rados.rados.0.smithi016.stdout:51: writing smithi01698494-51 from 1309629 to 1852295 tid 2 2023-01-25T05:22:27.034 INFO:tasks.rados.rados.0.smithi016.stdout:51: writing smithi01698494-51 from 2323973 to 2840350 tid 3 2023-01-25T05:22:27.034 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.034 INFO:tasks.rados.rados.0.smithi016.stdout:36: finishing write tid 3 to smithi01698494-36 2023-01-25T05:22:27.034 INFO:tasks.rados.rados.0.smithi016.stdout:36: finishing write tid 4 to smithi01698494-36 2023-01-25T05:22:27.035 INFO:tasks.rados.rados.0.smithi016.stdout:36: oid 36 updating version 0 to 6 2023-01-25T05:22:27.035 INFO:tasks.rados.rados.0.smithi016.stdout:36: oid 36 updating version 6 to 17 2023-01-25T05:22:27.035 INFO:tasks.rados.rados.0.smithi016.stdout:36: oid 36 updating version 17 to 18 2023-01-25T05:22:27.035 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 36 v 18 (ObjNum 35 snap 0 seq_num 35) dirty exists 2023-01-25T05:22:27.035 INFO:tasks.rados.rados.0.smithi016.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2023-01-25T05:22:27.036 INFO:tasks.rados.rados.0.smithi016.stdout:37: finishing write tid 1 to smithi01698494-37 2023-01-25T05:22:27.036 INFO:tasks.rados.rados.0.smithi016.stdout:37: finishing write tid 2 to smithi01698494-37 2023-01-25T05:22:27.036 INFO:tasks.rados.rados.0.smithi016.stdout:37: finishing write tid 3 to smithi01698494-37 2023-01-25T05:22:27.036 INFO:tasks.rados.rados.0.smithi016.stdout:37: finishing write tid 4 to smithi01698494-37 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:37: oid 37 updating version 0 to 8 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:37: oid 37 updating version 8 to 23 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:37: oid 37 updating version 23 to 24 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 37 v 24 (ObjNum 36 snap 0 seq_num 36) dirty exists 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2023-01-25T05:22:27.037 INFO:tasks.rados.rados.0.smithi016.stdout:38: finishing write tid 1 to smithi01698494-38 2023-01-25T05:22:27.038 INFO:tasks.rados.rados.0.smithi016.stdout:38: finishing write tid 2 to smithi01698494-38 2023-01-25T05:22:27.038 INFO:tasks.rados.rados.0.smithi016.stdout:38: finishing write tid 3 to smithi01698494-38 2023-01-25T05:22:27.038 INFO:tasks.rados.rados.0.smithi016.stdout:38: finishing write tid 4 to smithi01698494-38 2023-01-25T05:22:27.038 INFO:tasks.rados.rados.0.smithi016.stdout:38: oid 38 updating version 0 to 22 2023-01-25T05:22:27.039 INFO:tasks.rados.rados.0.smithi016.stdout:38: oid 38 updating version 22 to 23 2023-01-25T05:22:27.039 INFO:tasks.rados.rados.0.smithi016.stdout:38: oid 38 updating version 23 to 24 2023-01-25T05:22:27.039 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 38 v 24 (ObjNum 37 snap 0 seq_num 37) dirty exists 2023-01-25T05:22:27.039 INFO:tasks.rados.rados.0.smithi016.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2023-01-25T05:22:27.040 INFO:tasks.rados.rados.0.smithi016.stdout:39: finishing write tid 1 to smithi01698494-39 2023-01-25T05:22:27.040 INFO:tasks.rados.rados.0.smithi016.stdout:39: finishing write tid 2 to smithi01698494-39 2023-01-25T05:22:27.040 INFO:tasks.rados.rados.0.smithi016.stdout:39: finishing write tid 3 to smithi01698494-39 2023-01-25T05:22:27.040 INFO:tasks.rados.rados.0.smithi016.stdout:39: finishing write tid 4 to smithi01698494-39 2023-01-25T05:22:27.040 INFO:tasks.rados.rados.0.smithi016.stdout:39: oid 39 updating version 0 to 25 2023-01-25T05:22:27.041 INFO:tasks.rados.rados.0.smithi016.stdout:39: oid 39 updating version 25 to 27 2023-01-25T05:22:27.041 INFO:tasks.rados.rados.0.smithi016.stdout:39: oid 39 version 27 is already newer than 26 2023-01-25T05:22:27.041 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 39 v 27 (ObjNum 38 snap 0 seq_num 38) dirty exists 2023-01-25T05:22:27.041 INFO:tasks.rados.rados.0.smithi016.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2023-01-25T05:22:27.041 INFO:tasks.rados.rados.0.smithi016.stdout:40: finishing write tid 1 to smithi01698494-40 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: finishing write tid 2 to smithi01698494-40 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: finishing write tid 3 to smithi01698494-40 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: finishing write tid 4 to smithi01698494-40 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: oid 40 updating version 0 to 13 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: oid 40 version 13 is already newer than 11 2023-01-25T05:22:27.042 INFO:tasks.rados.rados.0.smithi016.stdout:40: oid 40 version 13 is already newer than 12 2023-01-25T05:22:27.043 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 40 v 13 (ObjNum 39 snap 0 seq_num 39) dirty exists 2023-01-25T05:22:27.043 INFO:tasks.rados.rados.0.smithi016.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2023-01-25T05:22:27.043 INFO:tasks.rados.rados.0.smithi016.stdout:41: finishing write tid 1 to smithi01698494-41 2023-01-25T05:22:27.043 INFO:tasks.rados.rados.0.smithi016.stdout:41: finishing write tid 2 to smithi01698494-41 2023-01-25T05:22:27.043 INFO:tasks.rados.rados.0.smithi016.stdout:41: finishing write tid 3 to smithi01698494-41 2023-01-25T05:22:27.044 INFO:tasks.rados.rados.0.smithi016.stdout:41: finishing write tid 4 to smithi01698494-41 2023-01-25T05:22:27.044 INFO:tasks.rados.rados.0.smithi016.stdout:41: oid 41 updating version 0 to 16 2023-01-25T05:22:27.044 INFO:tasks.rados.rados.0.smithi016.stdout:41: oid 41 version 16 is already newer than 15 2023-01-25T05:22:27.044 INFO:tasks.rados.rados.0.smithi016.stdout:41: oid 41 version 16 is already newer than 14 2023-01-25T05:22:27.044 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 41 v 16 (ObjNum 40 snap 0 seq_num 40) dirty exists 2023-01-25T05:22:27.045 INFO:tasks.rados.rados.0.smithi016.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2023-01-25T05:22:27.045 INFO:tasks.rados.rados.0.smithi016.stdout:42: finishing write tid 1 to smithi01698494-42 2023-01-25T05:22:27.045 INFO:tasks.rados.rados.0.smithi016.stdout:42: finishing write tid 2 to smithi01698494-42 2023-01-25T05:22:27.045 INFO:tasks.rados.rados.0.smithi016.stdout:42: finishing write tid 3 to smithi01698494-42 2023-01-25T05:22:27.045 INFO:tasks.rados.rados.0.smithi016.stdout:36: done (15 left) 2023-01-25T05:22:27.046 INFO:tasks.rados.rados.0.smithi016.stdout:37: done (14 left) 2023-01-25T05:22:27.046 INFO:tasks.rados.rados.0.smithi016.stdout:38: done (13 left) 2023-01-25T05:22:27.046 INFO:tasks.rados.rados.0.smithi016.stdout:39: done (12 left) 2023-01-25T05:22:27.046 INFO:tasks.rados.rados.0.smithi016.stdout:40: done (11 left) 2023-01-25T05:22:27.046 INFO:tasks.rados.rados.0.smithi016.stdout:41: done (10 left) 2023-01-25T05:22:27.047 INFO:tasks.rados.rados.0.smithi016.stdout:52: write initial oid 52 2023-01-25T05:22:27.047 INFO:tasks.rados.rados.0.smithi016.stdout:52: seq_num 51 ranges {457157=535817,1768111=402677} 2023-01-25T05:22:27.048 INFO:tasks.rados.rados.0.smithi016.stdout:52: writing smithi01698494-52 from 457157 to 992974 tid 1 2023-01-25T05:22:27.048 INFO:tasks.rados.rados.0.smithi016.stdout:52: writing smithi01698494-52 from 1768111 to 2170788 tid 2 2023-01-25T05:22:27.048 INFO:tasks.rados.rados.0.smithi016.stdout:53: write initial oid 53 2023-01-25T05:22:27.048 INFO:tasks.rados.rados.0.smithi016.stdout:53: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2023-01-25T05:22:27.049 INFO:tasks.rados.rados.0.smithi016.stdout:53: writing smithi01698494-53 from 473964 to 1085030 tid 1 2023-01-25T05:22:27.057 INFO:tasks.rados.rados.0.smithi016.stdout:53: writing smithi01698494-53 from 1826593 to 2372928 tid 2 2023-01-25T05:22:27.060 INFO:tasks.rados.rados.0.smithi016.stdout:53: writing smithi01698494-53 from 2778819 to 2873964 tid 3 2023-01-25T05:22:27.060 INFO:tasks.rados.rados.0.smithi016.stdout:54: write initial oid 54 2023-01-25T05:22:27.060 INFO:tasks.rados.rados.0.smithi016.stdout:54: seq_num 53 ranges {490771=686315,1885075=406346} 2023-01-25T05:22:27.066 INFO:tasks.rados.rados.0.smithi016.stdout:54: writing smithi01698494-54 from 490771 to 1177086 tid 1 2023-01-25T05:22:27.072 INFO:tasks.rados.rados.0.smithi016.stdout:54: writing smithi01698494-54 from 1885075 to 2291421 tid 2 2023-01-25T05:22:27.073 INFO:tasks.rados.rados.0.smithi016.stdout:55: write initial oid 55 2023-01-25T05:22:27.073 INFO:tasks.rados.rados.0.smithi016.stdout:55: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2023-01-25T05:22:27.078 INFO:tasks.rados.rados.0.smithi016.stdout:55: writing smithi01698494-55 from 507578 to 985495 tid 1 2023-01-25T05:22:27.084 INFO:tasks.rados.rados.0.smithi016.stdout:55: writing smithi01698494-55 from 1543557 to 2093561 tid 2 2023-01-25T05:22:27.088 INFO:tasks.rados.rados.0.smithi016.stdout:55: writing smithi01698494-55 from 2833665 to 2907578 tid 3 2023-01-25T05:22:27.089 INFO:tasks.rados.rados.0.smithi016.stdout:56: write initial oid 56 2023-01-25T05:22:27.089 INFO:tasks.rados.rados.0.smithi016.stdout:56: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2023-01-25T05:22:27.094 INFO:tasks.rados.rados.0.smithi016.stdout:56: writing smithi01698494-56 from 524385 to 1077551 tid 1 2023-01-25T05:22:27.099 INFO:tasks.rados.rados.0.smithi016.stdout:56: writing smithi01698494-56 from 1602039 to 2012054 tid 2 2023-01-25T05:22:27.105 INFO:tasks.rados.rados.0.smithi016.stdout:56: writing smithi01698494-56 from 2577441 to 2924385 tid 3 2023-01-25T05:22:27.105 INFO:tasks.rados.rados.0.smithi016.stdout:57: write initial oid 57 2023-01-25T05:22:27.105 INFO:tasks.rados.rados.0.smithi016.stdout:57: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2023-01-25T05:22:27.111 INFO:tasks.rados.rados.0.smithi016.stdout:57: writing smithi01698494-57 from 541192 to 1169607 tid 1 2023-01-25T05:22:27.117 INFO:tasks.rados.rados.0.smithi016.stdout:57: writing smithi01698494-57 from 1660521 to 2214194 tid 2 2023-01-25T05:22:27.121 INFO:tasks.rados.rados.0.smithi016.stdout:57: writing smithi01698494-57 from 2888511 to 2941192 tid 3 2023-01-25T05:22:27.121 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.122 INFO:tasks.rados.rados.0.smithi016.stdout:42: finishing write tid 4 to smithi01698494-42 2023-01-25T05:22:27.122 INFO:tasks.rados.rados.0.smithi016.stdout:42: oid 42 updating version 0 to 17 2023-01-25T05:22:27.122 INFO:tasks.rados.rados.0.smithi016.stdout:42: oid 42 updating version 17 to 18 2023-01-25T05:22:27.122 INFO:tasks.rados.rados.0.smithi016.stdout:42: oid 42 updating version 18 to 19 2023-01-25T05:22:27.122 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 42 v 19 (ObjNum 41 snap 0 seq_num 41) dirty exists 2023-01-25T05:22:27.123 INFO:tasks.rados.rados.0.smithi016.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2023-01-25T05:22:27.123 INFO:tasks.rados.rados.0.smithi016.stdout:42: done (15 left) 2023-01-25T05:22:27.123 INFO:tasks.rados.rados.0.smithi016.stdout:58: write initial oid 58 2023-01-25T05:22:27.123 INFO:tasks.rados.rados.0.smithi016.stdout:58: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2023-01-25T05:22:27.128 INFO:tasks.rados.rados.0.smithi016.stdout:58: writing smithi01698494-58 from 557999 to 1261663 tid 1 2023-01-25T05:22:27.133 INFO:tasks.rados.rados.0.smithi016.stdout:58: writing smithi01698494-58 from 1719003 to 2132687 tid 2 2023-01-25T05:22:27.138 INFO:tasks.rados.rados.0.smithi016.stdout:58: writing smithi01698494-58 from 2632287 to 2957999 tid 3 2023-01-25T05:22:27.139 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.139 INFO:tasks.rados.rados.0.smithi016.stdout:43: finishing write tid 1 to smithi01698494-43 2023-01-25T05:22:27.139 INFO:tasks.rados.rados.0.smithi016.stdout:43: finishing write tid 2 to smithi01698494-43 2023-01-25T05:22:27.139 INFO:tasks.rados.rados.0.smithi016.stdout:43: finishing write tid 3 to smithi01698494-43 2023-01-25T05:22:27.139 INFO:tasks.rados.rados.0.smithi016.stdout:43: finishing write tid 4 to smithi01698494-43 2023-01-25T05:22:27.140 INFO:tasks.rados.rados.0.smithi016.stdout:43: oid 43 updating version 0 to 20 2023-01-25T05:22:27.140 INFO:tasks.rados.rados.0.smithi016.stdout:43: oid 43 updating version 20 to 21 2023-01-25T05:22:27.140 INFO:tasks.rados.rados.0.smithi016.stdout:43: oid 43 version 21 is already newer than 7 2023-01-25T05:22:27.140 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 43 v 21 (ObjNum 42 snap 0 seq_num 42) dirty exists 2023-01-25T05:22:27.140 INFO:tasks.rados.rados.0.smithi016.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2023-01-25T05:22:27.141 INFO:tasks.rados.rados.0.smithi016.stdout:43: done (15 left) 2023-01-25T05:22:27.141 INFO:tasks.rados.rados.0.smithi016.stdout:59: write initial oid 59 2023-01-25T05:22:27.141 INFO:tasks.rados.rados.0.smithi016.stdout:59: seq_num 58 ranges {574806=778913,2061132=557342} 2023-01-25T05:22:27.145 INFO:tasks.rados.rados.0.smithi016.stdout:59: writing smithi01698494-59 from 574806 to 1353719 tid 1 2023-01-25T05:22:27.153 INFO:tasks.rados.rados.0.smithi016.stdout:59: writing smithi01698494-59 from 2061132 to 2618474 tid 2 2023-01-25T05:22:27.154 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.154 INFO:tasks.rados.rados.0.smithi016.stdout:44: finishing write tid 1 to smithi01698494-44 2023-01-25T05:22:27.154 INFO:tasks.rados.rados.0.smithi016.stdout:44: finishing write tid 2 to smithi01698494-44 2023-01-25T05:22:27.154 INFO:tasks.rados.rados.0.smithi016.stdout:44: finishing write tid 3 to smithi01698494-44 2023-01-25T05:22:27.154 INFO:tasks.rados.rados.0.smithi016.stdout:44: finishing write tid 4 to smithi01698494-44 2023-01-25T05:22:27.155 INFO:tasks.rados.rados.0.smithi016.stdout:44: oid 44 updating version 0 to 30 2023-01-25T05:22:27.155 INFO:tasks.rados.rados.0.smithi016.stdout:44: oid 44 version 30 is already newer than 29 2023-01-25T05:22:27.155 INFO:tasks.rados.rados.0.smithi016.stdout:44: oid 44 version 30 is already newer than 28 2023-01-25T05:22:27.155 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 44 v 30 (ObjNum 43 snap 0 seq_num 43) dirty exists 2023-01-25T05:22:27.155 INFO:tasks.rados.rados.0.smithi016.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2023-01-25T05:22:27.156 INFO:tasks.rados.rados.0.smithi016.stdout:45: finishing write tid 1 to smithi01698494-45 2023-01-25T05:22:27.156 INFO:tasks.rados.rados.0.smithi016.stdout:45: finishing write tid 2 to smithi01698494-45 2023-01-25T05:22:27.156 INFO:tasks.rados.rados.0.smithi016.stdout:45: finishing write tid 3 to smithi01698494-45 2023-01-25T05:22:27.156 INFO:tasks.rados.rados.0.smithi016.stdout:45: finishing write tid 4 to smithi01698494-45 2023-01-25T05:22:27.156 INFO:tasks.rados.rados.0.smithi016.stdout:45: oid 45 updating version 0 to 23 2023-01-25T05:22:27.157 INFO:tasks.rados.rados.0.smithi016.stdout:45: oid 45 updating version 23 to 24 2023-01-25T05:22:27.157 INFO:tasks.rados.rados.0.smithi016.stdout:45: oid 45 version 24 is already newer than 8 2023-01-25T05:22:27.157 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 45 v 24 (ObjNum 44 snap 0 seq_num 44) dirty exists 2023-01-25T05:22:27.157 INFO:tasks.rados.rados.0.smithi016.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2023-01-25T05:22:27.157 INFO:tasks.rados.rados.0.smithi016.stdout:46: finishing write tid 1 to smithi01698494-46 2023-01-25T05:22:27.158 INFO:tasks.rados.rados.0.smithi016.stdout:46: finishing write tid 2 to smithi01698494-46 2023-01-25T05:22:27.158 INFO:tasks.rados.rados.0.smithi016.stdout:46: finishing write tid 3 to smithi01698494-46 2023-01-25T05:22:27.158 INFO:tasks.rados.rados.0.smithi016.stdout:44: done (15 left) 2023-01-25T05:22:27.158 INFO:tasks.rados.rados.0.smithi016.stdout:45: done (14 left) 2023-01-25T05:22:27.158 INFO:tasks.rados.rados.0.smithi016.stdout:60: write initial oid 60 2023-01-25T05:22:27.159 INFO:tasks.rados.rados.0.smithi016.stdout:60: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2023-01-25T05:22:27.159 INFO:tasks.rados.rados.0.smithi016.stdout:60: writing smithi01698494-60 from 591613 to 1045775 tid 1 2023-01-25T05:22:27.164 INFO:tasks.rados.rados.0.smithi016.stdout:60: writing smithi01698494-60 from 1719614 to 2136967 tid 2 2023-01-25T05:22:27.169 INFO:tasks.rados.rados.0.smithi016.stdout:60: writing smithi01698494-60 from 2570780 to 2991613 tid 3 2023-01-25T05:22:27.170 INFO:tasks.rados.rados.0.smithi016.stdout:61: write initial oid 61 2023-01-25T05:22:27.170 INFO:tasks.rados.rados.0.smithi016.stdout:61: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2023-01-25T05:22:27.175 INFO:tasks.rados.rados.0.smithi016.stdout:61: writing smithi01698494-61 from 608420 to 1137831 tid 1 2023-01-25T05:22:27.182 INFO:tasks.rados.rados.0.smithi016.stdout:61: writing smithi01698494-61 from 1778096 to 2339107 tid 2 2023-01-25T05:22:27.186 INFO:tasks.rados.rados.0.smithi016.stdout:61: writing smithi01698494-61 from 2881850 to 3008420 tid 3 2023-01-25T05:22:27.186 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.187 INFO:tasks.rados.rados.0.smithi016.stdout:46: finishing write tid 4 to smithi01698494-46 2023-01-25T05:22:27.187 INFO:tasks.rados.rados.0.smithi016.stdout:46: oid 46 updating version 0 to 33 2023-01-25T05:22:27.187 INFO:tasks.rados.rados.0.smithi016.stdout:46: oid 46 version 33 is already newer than 31 2023-01-25T05:22:27.187 INFO:tasks.rados.rados.0.smithi016.stdout:46: oid 46 version 33 is already newer than 32 2023-01-25T05:22:27.187 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 46 v 33 (ObjNum 45 snap 0 seq_num 45) dirty exists 2023-01-25T05:22:27.188 INFO:tasks.rados.rados.0.smithi016.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2023-01-25T05:22:27.188 INFO:tasks.rados.rados.0.smithi016.stdout:46: done (15 left) 2023-01-25T05:22:27.188 INFO:tasks.rados.rados.0.smithi016.stdout:62: write initial oid 62 2023-01-25T05:22:27.188 INFO:tasks.rados.rados.0.smithi016.stdout:62: seq_num 61 ranges {625227=721013,1952931=704669} 2023-01-25T05:22:27.193 INFO:tasks.rados.rados.0.smithi016.stdout:62: writing smithi01698494-62 from 625227 to 1346240 tid 1 2023-01-25T05:22:27.201 INFO:tasks.rados.rados.0.smithi016.stdout:62: writing smithi01698494-62 from 1952931 to 2657600 tid 2 2023-01-25T05:22:27.201 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.201 INFO:tasks.rados.rados.0.smithi016.stdout:47: finishing write tid 1 to smithi01698494-47 2023-01-25T05:22:27.202 INFO:tasks.rados.rados.0.smithi016.stdout:47: finishing write tid 2 to smithi01698494-47 2023-01-25T05:22:27.202 INFO:tasks.rados.rados.0.smithi016.stdout:47: finishing write tid 3 to smithi01698494-47 2023-01-25T05:22:27.202 INFO:tasks.rados.rados.0.smithi016.stdout:47: finishing write tid 4 to smithi01698494-47 2023-01-25T05:22:27.202 INFO:tasks.rados.rados.0.smithi016.stdout:47: oid 47 updating version 0 to 35 2023-01-25T05:22:27.202 INFO:tasks.rados.rados.0.smithi016.stdout:47: oid 47 updating version 35 to 36 2023-01-25T05:22:27.203 INFO:tasks.rados.rados.0.smithi016.stdout:47: oid 47 version 36 is already newer than 34 2023-01-25T05:22:27.203 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 47 v 36 (ObjNum 46 snap 0 seq_num 46) dirty exists 2023-01-25T05:22:27.203 INFO:tasks.rados.rados.0.smithi016.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2023-01-25T05:22:27.203 INFO:tasks.rados.rados.0.smithi016.stdout:48: finishing write tid 1 to smithi01698494-48 2023-01-25T05:22:27.203 INFO:tasks.rados.rados.0.smithi016.stdout:47: done (15 left) 2023-01-25T05:22:27.204 INFO:tasks.rados.rados.0.smithi016.stdout:63: write initial oid 63 2023-01-25T05:22:27.204 INFO:tasks.rados.rados.0.smithi016.stdout:63: seq_num 62 ranges {642034=796262,1895060=564680} 2023-01-25T05:22:27.208 INFO:tasks.rados.rados.0.smithi016.stdout:63: writing smithi01698494-63 from 642034 to 1438296 tid 1 2023-01-25T05:22:27.214 INFO:tasks.rados.rados.0.smithi016.stdout:63: writing smithi01698494-63 from 1895060 to 2459740 tid 2 2023-01-25T05:22:27.214 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.215 INFO:tasks.rados.rados.0.smithi016.stdout:48: finishing write tid 2 to smithi01698494-48 2023-01-25T05:22:27.215 INFO:tasks.rados.rados.0.smithi016.stdout:48: finishing write tid 3 to smithi01698494-48 2023-01-25T05:22:27.215 INFO:tasks.rados.rados.0.smithi016.stdout:48: finishing write tid 4 to smithi01698494-48 2023-01-25T05:22:27.215 INFO:tasks.rados.rados.0.smithi016.stdout:48: oid 48 updating version 0 to 37 2023-01-25T05:22:27.215 INFO:tasks.rados.rados.0.smithi016.stdout:48: oid 48 updating version 37 to 39 2023-01-25T05:22:27.216 INFO:tasks.rados.rados.0.smithi016.stdout:48: oid 48 version 39 is already newer than 38 2023-01-25T05:22:27.216 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 48 v 39 (ObjNum 47 snap 0 seq_num 47) dirty exists 2023-01-25T05:22:27.216 INFO:tasks.rados.rados.0.smithi016.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2023-01-25T05:22:27.216 INFO:tasks.rados.rados.0.smithi016.stdout:49: finishing write tid 1 to smithi01698494-49 2023-01-25T05:22:27.217 INFO:tasks.rados.rados.0.smithi016.stdout:48: done (15 left) 2023-01-25T05:22:27.217 INFO:tasks.rados.rados.0.smithi016.stdout:64: write initial oid 64 2023-01-25T05:22:27.217 INFO:tasks.rados.rados.0.smithi016.stdout:64: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2023-01-25T05:22:27.220 INFO:tasks.rados.rados.0.smithi016.stdout:64: writing smithi01698494-64 from 658841 to 1130352 tid 1 2023-01-25T05:22:27.227 INFO:tasks.rados.rados.0.smithi016.stdout:64: writing smithi01698494-64 from 1553542 to 2261880 tid 2 2023-01-25T05:22:27.232 INFO:tasks.rados.rados.0.smithi016.stdout:64: writing smithi01698494-64 from 2964119 to 3058841 tid 3 2023-01-25T05:22:27.232 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.232 INFO:tasks.rados.rados.0.smithi016.stdout:49: finishing write tid 2 to smithi01698494-49 2023-01-25T05:22:27.233 INFO:tasks.rados.rados.0.smithi016.stdout:49: finishing write tid 3 to smithi01698494-49 2023-01-25T05:22:27.233 INFO:tasks.rados.rados.0.smithi016.stdout:49: finishing write tid 4 to smithi01698494-49 2023-01-25T05:22:27.233 INFO:tasks.rados.rados.0.smithi016.stdout:49: oid 49 updating version 0 to 21 2023-01-25T05:22:27.233 INFO:tasks.rados.rados.0.smithi016.stdout:49: oid 49 version 21 is already newer than 20 2023-01-25T05:22:27.234 INFO:tasks.rados.rados.0.smithi016.stdout:49: oid 49 updating version 21 to 22 2023-01-25T05:22:27.234 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 49 v 22 (ObjNum 48 snap 0 seq_num 48) dirty exists 2023-01-25T05:22:27.234 INFO:tasks.rados.rados.0.smithi016.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2023-01-25T05:22:27.234 INFO:tasks.rados.rados.0.smithi016.stdout:50: finishing write tid 1 to smithi01698494-50 2023-01-25T05:22:27.234 INFO:tasks.rados.rados.0.smithi016.stdout:50: finishing write tid 2 to smithi01698494-50 2023-01-25T05:22:27.235 INFO:tasks.rados.rados.0.smithi016.stdout:50: finishing write tid 3 to smithi01698494-50 2023-01-25T05:22:27.235 INFO:tasks.rados.rados.0.smithi016.stdout:50: finishing write tid 4 to smithi01698494-50 2023-01-25T05:22:27.235 INFO:tasks.rados.rados.0.smithi016.stdout:50: oid 50 updating version 0 to 23 2023-01-25T05:22:27.235 INFO:tasks.rados.rados.0.smithi016.stdout:50: oid 50 updating version 23 to 25 2023-01-25T05:22:27.235 INFO:tasks.rados.rados.0.smithi016.stdout:50: oid 50 version 25 is already newer than 24 2023-01-25T05:22:27.236 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 50 v 25 (ObjNum 49 snap 0 seq_num 49) dirty exists 2023-01-25T05:22:27.236 INFO:tasks.rados.rados.0.smithi016.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2023-01-25T05:22:27.236 INFO:tasks.rados.rados.0.smithi016.stdout:51: finishing write tid 1 to smithi01698494-51 2023-01-25T05:22:27.236 INFO:tasks.rados.rados.0.smithi016.stdout:49: done (15 left) 2023-01-25T05:22:27.236 INFO:tasks.rados.rados.0.smithi016.stdout:50: done (14 left) 2023-01-25T05:22:27.237 INFO:tasks.rados.rados.0.smithi016.stdout:65: write initial oid 65 2023-01-25T05:22:27.237 INFO:tasks.rados.rados.0.smithi016.stdout:65: seq_num 64 ranges {675648=546760,2012024=568349} 2023-01-25T05:22:27.238 INFO:tasks.rados.rados.0.smithi016.stdout:65: writing smithi01698494-65 from 675648 to 1222408 tid 1 2023-01-25T05:22:27.245 INFO:tasks.rados.rados.0.smithi016.stdout:65: writing smithi01698494-65 from 2012024 to 2580373 tid 2 2023-01-25T05:22:27.246 INFO:tasks.rados.rados.0.smithi016.stdout:66: write initial oid 66 2023-01-25T05:22:27.246 INFO:tasks.rados.rados.0.smithi016.stdout:66: seq_num 65 ranges {692455=622009,2070506=712007} 2023-01-25T05:22:27.253 INFO:tasks.rados.rados.0.smithi016.stdout:66: writing smithi01698494-66 from 692455 to 1314464 tid 1 2023-01-25T05:22:27.261 INFO:tasks.rados.rados.0.smithi016.stdout:66: writing smithi01698494-66 from 2070506 to 2782513 tid 2 2023-01-25T05:22:27.262 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.262 INFO:tasks.rados.rados.0.smithi016.stdout:51: finishing write tid 2 to smithi01698494-51 2023-01-25T05:22:27.262 INFO:tasks.rados.rados.0.smithi016.stdout:51: finishing write tid 3 to smithi01698494-51 2023-01-25T05:22:27.262 INFO:tasks.rados.rados.0.smithi016.stdout:51: finishing write tid 4 to smithi01698494-51 2023-01-25T05:22:27.262 INFO:tasks.rados.rados.0.smithi016.stdout:52: finishing write tid 1 to smithi01698494-52 2023-01-25T05:22:27.263 INFO:tasks.rados.rados.0.smithi016.stdout:51: finishing write tid 5 to smithi01698494-51 2023-01-25T05:22:27.263 INFO:tasks.rados.rados.0.smithi016.stdout:51: oid 51 updating version 0 to 10 2023-01-25T05:22:27.263 INFO:tasks.rados.rados.0.smithi016.stdout:51: oid 51 updating version 10 to 27 2023-01-25T05:22:27.263 INFO:tasks.rados.rados.0.smithi016.stdout:51: oid 51 version 27 is already newer than 26 2023-01-25T05:22:27.263 INFO:tasks.rados.rados.0.smithi016.stdout:51: oid 51 version 27 is already newer than 9 2023-01-25T05:22:27.264 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 51 v 27 (ObjNum 50 snap 0 seq_num 50) dirty exists 2023-01-25T05:22:27.264 INFO:tasks.rados.rados.0.smithi016.stdout:51: left oid 51 (ObjNum 50 snap 0 seq_num 50) 2023-01-25T05:22:27.264 INFO:tasks.rados.rados.0.smithi016.stdout:52: finishing write tid 2 to smithi01698494-52 2023-01-25T05:22:27.264 INFO:tasks.rados.rados.0.smithi016.stdout:51: done (15 left) 2023-01-25T05:22:27.264 INFO:tasks.rados.rados.0.smithi016.stdout:67: write initial oid 67 2023-01-25T05:22:27.265 INFO:tasks.rados.rados.0.smithi016.stdout:67: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2023-01-25T05:22:27.268 INFO:tasks.rados.rados.0.smithi016.stdout:67: writing smithi01698494-67 from 709262 to 1406520 tid 1 2023-01-25T05:22:27.275 INFO:tasks.rados.rados.0.smithi016.stdout:67: writing smithi01698494-67 from 2012635 to 2584653 tid 2 2023-01-25T05:22:27.279 INFO:tasks.rados.rados.0.smithi016.stdout:67: writing smithi01698494-67 from 3046388 to 3109262 tid 3 2023-01-25T05:22:27.280 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.280 INFO:tasks.rados.rados.0.smithi016.stdout:52: finishing write tid 3 to smithi01698494-52 2023-01-25T05:22:27.280 INFO:tasks.rados.rados.0.smithi016.stdout:52: finishing write tid 4 to smithi01698494-52 2023-01-25T05:22:27.281 INFO:tasks.rados.rados.0.smithi016.stdout:52: oid 52 updating version 0 to 27 2023-01-25T05:22:27.281 INFO:tasks.rados.rados.0.smithi016.stdout:52: oid 52 version 27 is already newer than 9 2023-01-25T05:22:27.281 INFO:tasks.rados.rados.0.smithi016.stdout:52: oid 52 version 27 is already newer than 26 2023-01-25T05:22:27.281 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 52 v 27 (ObjNum 51 snap 0 seq_num 51) dirty exists 2023-01-25T05:22:27.282 INFO:tasks.rados.rados.0.smithi016.stdout:52: left oid 52 (ObjNum 51 snap 0 seq_num 51) 2023-01-25T05:22:27.282 INFO:tasks.rados.rados.0.smithi016.stdout:53: finishing write tid 1 to smithi01698494-53 2023-01-25T05:22:27.282 INFO:tasks.rados.rados.0.smithi016.stdout:53: finishing write tid 2 to smithi01698494-53 2023-01-25T05:22:27.283 INFO:tasks.rados.rados.0.smithi016.stdout:52: done (15 left) 2023-01-25T05:22:27.283 INFO:tasks.rados.rados.0.smithi016.stdout:68: write initial oid 68 2023-01-25T05:22:27.283 INFO:tasks.rados.rados.0.smithi016.stdout:68: seq_num 67 ranges {726069=772507,2071117=715676} 2023-01-25T05:22:27.286 INFO:tasks.rados.rados.0.smithi016.stdout:68: writing smithi01698494-68 from 726069 to 1498576 tid 1 2023-01-25T05:22:27.294 INFO:tasks.rados.rados.0.smithi016.stdout:68: writing smithi01698494-68 from 2071117 to 2786793 tid 2 2023-01-25T05:22:27.295 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.295 INFO:tasks.rados.rados.0.smithi016.stdout:53: finishing write tid 3 to smithi01698494-53 2023-01-25T05:22:27.295 INFO:tasks.rados.rados.0.smithi016.stdout:53: finishing write tid 4 to smithi01698494-53 2023-01-25T05:22:27.295 INFO:tasks.rados.rados.0.smithi016.stdout:53: finishing write tid 5 to smithi01698494-53 2023-01-25T05:22:27.296 INFO:tasks.rados.rados.0.smithi016.stdout:53: oid 53 updating version 0 to 43 2023-01-25T05:22:27.296 INFO:tasks.rados.rados.0.smithi016.stdout:53: oid 53 version 43 is already newer than 41 2023-01-25T05:22:27.296 INFO:tasks.rados.rados.0.smithi016.stdout:53: oid 53 version 43 is already newer than 40 2023-01-25T05:22:27.296 INFO:tasks.rados.rados.0.smithi016.stdout:53: oid 53 version 43 is already newer than 42 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 53 v 43 (ObjNum 52 snap 0 seq_num 52) dirty exists 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:53: left oid 53 (ObjNum 52 snap 0 seq_num 52) 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:54: finishing write tid 1 to smithi01698494-54 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:54: finishing write tid 2 to smithi01698494-54 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:54: finishing write tid 3 to smithi01698494-54 2023-01-25T05:22:27.297 INFO:tasks.rados.rados.0.smithi016.stdout:54: finishing write tid 4 to smithi01698494-54 2023-01-25T05:22:27.298 INFO:tasks.rados.rados.0.smithi016.stdout:54: oid 54 updating version 0 to 30 2023-01-25T05:22:27.298 INFO:tasks.rados.rados.0.smithi016.stdout:54: oid 54 version 30 is already newer than 29 2023-01-25T05:22:27.298 INFO:tasks.rados.rados.0.smithi016.stdout:54: oid 54 version 30 is already newer than 10 2023-01-25T05:22:27.298 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 54 v 30 (ObjNum 53 snap 0 seq_num 53) dirty exists 2023-01-25T05:22:27.298 INFO:tasks.rados.rados.0.smithi016.stdout:54: left oid 54 (ObjNum 53 snap 0 seq_num 53) 2023-01-25T05:22:27.299 INFO:tasks.rados.rados.0.smithi016.stdout:55: finishing write tid 1 to smithi01698494-55 2023-01-25T05:22:27.299 INFO:tasks.rados.rados.0.smithi016.stdout:55: finishing write tid 2 to smithi01698494-55 2023-01-25T05:22:27.299 INFO:tasks.rados.rados.0.smithi016.stdout:55: finishing write tid 3 to smithi01698494-55 2023-01-25T05:22:27.299 INFO:tasks.rados.rados.0.smithi016.stdout:55: finishing write tid 4 to smithi01698494-55 2023-01-25T05:22:27.299 INFO:tasks.rados.rados.0.smithi016.stdout:55: finishing write tid 5 to smithi01698494-55 2023-01-25T05:22:27.300 INFO:tasks.rados.rados.0.smithi016.stdout:55: oid 55 updating version 0 to 47 2023-01-25T05:22:27.300 INFO:tasks.rados.rados.0.smithi016.stdout:55: oid 55 version 47 is already newer than 46 2023-01-25T05:22:27.300 INFO:tasks.rados.rados.0.smithi016.stdout:55: oid 55 version 47 is already newer than 45 2023-01-25T05:22:27.301 INFO:tasks.rados.rados.0.smithi016.stdout:55: oid 55 version 47 is already newer than 44 2023-01-25T05:22:27.301 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 55 v 47 (ObjNum 54 snap 0 seq_num 54) dirty exists 2023-01-25T05:22:27.301 INFO:tasks.rados.rados.0.smithi016.stdout:55: left oid 55 (ObjNum 54 snap 0 seq_num 54) 2023-01-25T05:22:27.303 INFO:tasks.rados.rados.0.smithi016.stdout:56: finishing write tid 1 to smithi01698494-56 2023-01-25T05:22:27.304 INFO:tasks.rados.rados.0.smithi016.stdout:53: done (15 left) 2023-01-25T05:22:27.304 INFO:tasks.rados.rados.0.smithi016.stdout:54: done (14 left) 2023-01-25T05:22:27.304 INFO:tasks.rados.rados.0.smithi016.stdout:55: done (13 left) 2023-01-25T05:22:27.304 INFO:tasks.rados.rados.0.smithi016.stdout:69: write initial oid 69 2023-01-25T05:22:27.305 INFO:tasks.rados.rados.0.smithi016.stdout:69: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2023-01-25T05:22:27.306 INFO:tasks.rados.rados.0.smithi016.stdout:69: writing smithi01698494-69 from 742876 to 1190632 tid 1 2023-01-25T05:22:27.306 INFO:tasks.rados.rados.0.smithi016.stdout:69: writing smithi01698494-69 from 1729599 to 2305286 tid 2 2023-01-25T05:22:27.311 INFO:tasks.rados.rados.0.smithi016.stdout:69: writing smithi01698494-69 from 3101234 to 3142876 tid 3 2023-01-25T05:22:27.311 INFO:tasks.rados.rados.0.smithi016.stdout:70: write initial oid 70 2023-01-25T05:22:27.311 INFO:tasks.rados.rados.0.smithi016.stdout:70: seq_num 69 ranges {759683=639358,1904434=719345,3128657=31026} 2023-01-25T05:22:27.318 INFO:tasks.rados.rados.0.smithi016.stdout:70: writing smithi01698494-70 from 759683 to 1399041 tid 1 2023-01-25T05:22:27.325 INFO:tasks.rados.rados.0.smithi016.stdout:70: writing smithi01698494-70 from 1904434 to 2623779 tid 2 2023-01-25T05:22:27.328 INFO:tasks.rados.rados.0.smithi016.stdout:70: writing smithi01698494-70 from 3128657 to 3159683 tid 3 2023-01-25T05:22:27.329 INFO:tasks.rados.rados.0.smithi016.stdout:71: write initial oid 71 2023-01-25T05:22:27.329 INFO:tasks.rados.rados.0.smithi016.stdout:71: seq_num 70 ranges {776490=714607,2246563=579356} 2023-01-25T05:22:27.337 INFO:tasks.rados.rados.0.smithi016.stdout:71: writing smithi01698494-71 from 776490 to 1491097 tid 1 2023-01-25T05:22:27.344 INFO:tasks.rados.rados.0.smithi016.stdout:71: writing smithi01698494-71 from 2246563 to 2825919 tid 2 2023-01-25T05:22:27.345 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.345 INFO:tasks.rados.rados.0.smithi016.stdout:56: finishing write tid 2 to smithi01698494-56 2023-01-25T05:22:27.345 INFO:tasks.rados.rados.0.smithi016.stdout:56: finishing write tid 3 to smithi01698494-56 2023-01-25T05:22:27.345 INFO:tasks.rados.rados.0.smithi016.stdout:56: finishing write tid 4 to smithi01698494-56 2023-01-25T05:22:27.345 INFO:tasks.rados.rados.0.smithi016.stdout:56: finishing write tid 5 to smithi01698494-56 2023-01-25T05:22:27.346 INFO:tasks.rados.rados.0.smithi016.stdout:56: oid 56 updating version 0 to 29 2023-01-25T05:22:27.346 INFO:tasks.rados.rados.0.smithi016.stdout:56: oid 56 version 29 is already newer than 11 2023-01-25T05:22:27.346 INFO:tasks.rados.rados.0.smithi016.stdout:56: oid 56 updating version 29 to 31 2023-01-25T05:22:27.346 INFO:tasks.rados.rados.0.smithi016.stdout:56: oid 56 version 31 is already newer than 30 2023-01-25T05:22:27.346 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 56 v 31 (ObjNum 55 snap 0 seq_num 55) dirty exists 2023-01-25T05:22:27.347 INFO:tasks.rados.rados.0.smithi016.stdout:56: left oid 56 (ObjNum 55 snap 0 seq_num 55) 2023-01-25T05:22:27.347 INFO:tasks.rados.rados.0.smithi016.stdout:57: finishing write tid 1 to smithi01698494-57 2023-01-25T05:22:27.347 INFO:tasks.rados.rados.0.smithi016.stdout:56: done (15 left) 2023-01-25T05:22:27.347 INFO:tasks.rados.rados.0.smithi016.stdout:72: write initial oid 72 2023-01-25T05:22:27.347 INFO:tasks.rados.rados.0.smithi016.stdout:72: seq_num 71 ranges {793297=789856,2305045=723014} 2023-01-25T05:22:27.352 INFO:tasks.rados.rados.0.smithi016.stdout:72: writing smithi01698494-72 from 793297 to 1583153 tid 1 2023-01-25T05:22:27.361 INFO:tasks.rados.rados.0.smithi016.stdout:72: writing smithi01698494-72 from 2305045 to 3028059 tid 2 2023-01-25T05:22:27.361 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.361 INFO:tasks.rados.rados.0.smithi016.stdout:57: finishing write tid 2 to smithi01698494-57 2023-01-25T05:22:27.361 INFO:tasks.rados.rados.0.smithi016.stdout:57: finishing write tid 3 to smithi01698494-57 2023-01-25T05:22:27.362 INFO:tasks.rados.rados.0.smithi016.stdout:57: finishing write tid 4 to smithi01698494-57 2023-01-25T05:22:27.362 INFO:tasks.rados.rados.0.smithi016.stdout:57: finishing write tid 5 to smithi01698494-57 2023-01-25T05:22:27.362 INFO:tasks.rados.rados.0.smithi016.stdout:57: oid 57 updating version 0 to 26 2023-01-25T05:22:27.362 INFO:tasks.rados.rados.0.smithi016.stdout:57: oid 57 updating version 26 to 33 2023-01-25T05:22:27.362 INFO:tasks.rados.rados.0.smithi016.stdout:57: oid 57 version 33 is already newer than 32 2023-01-25T05:22:27.363 INFO:tasks.rados.rados.0.smithi016.stdout:57: oid 57 version 33 is already newer than 27 2023-01-25T05:22:27.363 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 57 v 33 (ObjNum 56 snap 0 seq_num 56) dirty exists 2023-01-25T05:22:27.363 INFO:tasks.rados.rados.0.smithi016.stdout:57: left oid 57 (ObjNum 56 snap 0 seq_num 56) 2023-01-25T05:22:27.363 INFO:tasks.rados.rados.0.smithi016.stdout:58: finishing write tid 1 to smithi01698494-58 2023-01-25T05:22:27.364 INFO:tasks.rados.rados.0.smithi016.stdout:57: done (15 left) 2023-01-25T05:22:27.364 INFO:tasks.rados.rados.0.smithi016.stdout:73: write initial oid 73 2023-01-25T05:22:27.364 INFO:tasks.rados.rados.0.smithi016.stdout:73: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2023-01-25T05:22:27.365 INFO:tasks.rados.rados.0.smithi016.stdout:73: writing smithi01698494-73 from 410104 to 875209 tid 1 2023-01-25T05:22:27.373 INFO:tasks.rados.rados.0.smithi016.stdout:73: writing smithi01698494-73 from 1563527 to 2146552 tid 2 2023-01-25T05:22:27.380 INFO:tasks.rados.rados.0.smithi016.stdout:73: writing smithi01698494-73 from 2810926 to 3210104 tid 3 2023-01-25T05:22:27.381 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.381 INFO:tasks.rados.rados.0.smithi016.stdout:58: finishing write tid 2 to smithi01698494-58 2023-01-25T05:22:27.381 INFO:tasks.rados.rados.0.smithi016.stdout:58: finishing write tid 3 to smithi01698494-58 2023-01-25T05:22:27.382 INFO:tasks.rados.rados.0.smithi016.stdout:58: finishing write tid 4 to smithi01698494-58 2023-01-25T05:22:27.382 INFO:tasks.rados.rados.0.smithi016.stdout:58: finishing write tid 5 to smithi01698494-58 2023-01-25T05:22:27.382 INFO:tasks.rados.rados.0.smithi016.stdout:58: oid 58 updating version 0 to 36 2023-01-25T05:22:27.382 INFO:tasks.rados.rados.0.smithi016.stdout:58: oid 58 updating version 36 to 37 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:58: oid 58 version 37 is already newer than 28 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:58: oid 58 version 37 is already newer than 35 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 58 v 37 (ObjNum 57 snap 0 seq_num 57) dirty exists 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:58: left oid 58 (ObjNum 57 snap 0 seq_num 57) 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:59: finishing write tid 1 to smithi01698494-59 2023-01-25T05:22:27.383 INFO:tasks.rados.rados.0.smithi016.stdout:59: finishing write tid 2 to smithi01698494-59 2023-01-25T05:22:27.384 INFO:tasks.rados.rados.0.smithi016.stdout:59: finishing write tid 3 to smithi01698494-59 2023-01-25T05:22:27.384 INFO:tasks.rados.rados.0.smithi016.stdout:59: finishing write tid 4 to smithi01698494-59 2023-01-25T05:22:27.384 INFO:tasks.rados.rados.0.smithi016.stdout:59: oid 59 updating version 0 to 29 2023-01-25T05:22:27.384 INFO:tasks.rados.rados.0.smithi016.stdout:59: oid 59 updating version 29 to 39 2023-01-25T05:22:27.384 INFO:tasks.rados.rados.0.smithi016.stdout:59: oid 59 updating version 39 to 40 2023-01-25T05:22:27.385 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 59 v 40 (ObjNum 58 snap 0 seq_num 58) dirty exists 2023-01-25T05:22:27.385 INFO:tasks.rados.rados.0.smithi016.stdout:59: left oid 59 (ObjNum 58 snap 0 seq_num 58) 2023-01-25T05:22:27.385 INFO:tasks.rados.rados.0.smithi016.stdout:60: finishing write tid 1 to smithi01698494-60 2023-01-25T05:22:27.385 INFO:tasks.rados.rados.0.smithi016.stdout:60: finishing write tid 2 to smithi01698494-60 2023-01-25T05:22:27.385 INFO:tasks.rados.rados.0.smithi016.stdout:60: finishing write tid 3 to smithi01698494-60 2023-01-25T05:22:27.386 INFO:tasks.rados.rados.0.smithi016.stdout:60: finishing write tid 4 to smithi01698494-60 2023-01-25T05:22:27.386 INFO:tasks.rados.rados.0.smithi016.stdout:60: finishing write tid 5 to smithi01698494-60 2023-01-25T05:22:27.386 INFO:tasks.rados.rados.0.smithi016.stdout:60: oid 60 updating version 0 to 33 2023-01-25T05:22:27.387 INFO:tasks.rados.rados.0.smithi016.stdout:60: oid 60 updating version 33 to 34 2023-01-25T05:22:27.387 INFO:tasks.rados.rados.0.smithi016.stdout:60: oid 60 updating version 34 to 35 2023-01-25T05:22:27.387 INFO:tasks.rados.rados.0.smithi016.stdout:60: oid 60 version 35 is already newer than 12 2023-01-25T05:22:27.387 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 60 v 35 (ObjNum 59 snap 0 seq_num 59) dirty exists 2023-01-25T05:22:27.387 INFO:tasks.rados.rados.0.smithi016.stdout:60: left oid 60 (ObjNum 59 snap 0 seq_num 59) 2023-01-25T05:22:27.388 INFO:tasks.rados.rados.0.smithi016.stdout:61: finishing write tid 1 to smithi01698494-61 2023-01-25T05:22:27.388 INFO:tasks.rados.rados.0.smithi016.stdout:61: finishing write tid 2 to smithi01698494-61 2023-01-25T05:22:27.388 INFO:tasks.rados.rados.0.smithi016.stdout:61: finishing write tid 3 to smithi01698494-61 2023-01-25T05:22:27.388 INFO:tasks.rados.rados.0.smithi016.stdout:61: finishing write tid 4 to smithi01698494-61 2023-01-25T05:22:27.388 INFO:tasks.rados.rados.0.smithi016.stdout:61: finishing write tid 5 to smithi01698494-61 2023-01-25T05:22:27.389 INFO:tasks.rados.rados.0.smithi016.stdout:61: oid 61 updating version 0 to 11 2023-01-25T05:22:27.389 INFO:tasks.rados.rados.0.smithi016.stdout:61: oid 61 updating version 11 to 32 2023-01-25T05:22:27.389 INFO:tasks.rados.rados.0.smithi016.stdout:61: oid 61 updating version 32 to 34 2023-01-25T05:22:27.389 INFO:tasks.rados.rados.0.smithi016.stdout:61: oid 61 version 34 is already newer than 33 2023-01-25T05:22:27.390 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 61 v 34 (ObjNum 60 snap 0 seq_num 60) dirty exists 2023-01-25T05:22:27.390 INFO:tasks.rados.rados.0.smithi016.stdout:61: left oid 61 (ObjNum 60 snap 0 seq_num 60) 2023-01-25T05:22:27.390 INFO:tasks.rados.rados.0.smithi016.stdout:62: finishing write tid 1 to smithi01698494-62 2023-01-25T05:22:27.390 INFO:tasks.rados.rados.0.smithi016.stdout:62: finishing write tid 2 to smithi01698494-62 2023-01-25T05:22:27.390 INFO:tasks.rados.rados.0.smithi016.stdout:62: finishing write tid 3 to smithi01698494-62 2023-01-25T05:22:27.391 INFO:tasks.rados.rados.0.smithi016.stdout:62: finishing write tid 4 to smithi01698494-62 2023-01-25T05:22:27.391 INFO:tasks.rados.rados.0.smithi016.stdout:62: oid 62 updating version 0 to 13 2023-01-25T05:22:27.391 INFO:tasks.rados.rados.0.smithi016.stdout:62: oid 62 updating version 13 to 37 2023-01-25T05:22:27.391 INFO:tasks.rados.rados.0.smithi016.stdout:62: oid 62 updating version 37 to 38 2023-01-25T05:22:27.391 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 62 v 38 (ObjNum 61 snap 0 seq_num 61) dirty exists 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:62: left oid 62 (ObjNum 61 snap 0 seq_num 61) 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:63: finishing write tid 1 to smithi01698494-63 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:63: finishing write tid 2 to smithi01698494-63 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:63: finishing write tid 3 to smithi01698494-63 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:64: finishing write tid 1 to smithi01698494-64 2023-01-25T05:22:27.392 INFO:tasks.rados.rados.0.smithi016.stdout:64: finishing write tid 2 to smithi01698494-64 2023-01-25T05:22:27.393 INFO:tasks.rados.rados.0.smithi016.stdout:64: finishing write tid 3 to smithi01698494-64 2023-01-25T05:22:27.393 INFO:tasks.rados.rados.0.smithi016.stdout:64: finishing write tid 4 to smithi01698494-64 2023-01-25T05:22:27.393 INFO:tasks.rados.rados.0.smithi016.stdout:64: finishing write tid 5 to smithi01698494-64 2023-01-25T05:22:27.393 INFO:tasks.rados.rados.0.smithi016.stdout:64: oid 64 updating version 0 to 42 2023-01-25T05:22:27.393 INFO:tasks.rados.rados.0.smithi016.stdout:64: oid 64 version 42 is already newer than 14 2023-01-25T05:22:27.394 INFO:tasks.rados.rados.0.smithi016.stdout:64: oid 64 version 42 is already newer than 41 2023-01-25T05:22:27.394 INFO:tasks.rados.rados.0.smithi016.stdout:64: oid 64 version 42 is already newer than 40 2023-01-25T05:22:27.394 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 64 v 42 (ObjNum 63 snap 0 seq_num 63) dirty exists 2023-01-25T05:22:27.394 INFO:tasks.rados.rados.0.smithi016.stdout:64: left oid 64 (ObjNum 63 snap 0 seq_num 63) 2023-01-25T05:22:27.394 INFO:tasks.rados.rados.0.smithi016.stdout:58: done (15 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:59: done (14 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:60: done (13 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:61: done (12 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:62: done (11 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:64: done (10 left) 2023-01-25T05:22:27.395 INFO:tasks.rados.rados.0.smithi016.stdout:74: write initial oid 74 2023-01-25T05:22:27.396 INFO:tasks.rados.rados.0.smithi016.stdout:74: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2023-01-25T05:22:27.397 INFO:tasks.rados.rados.0.smithi016.stdout:74: writing smithi01698494-74 from 426911 to 967265 tid 1 2023-01-25T05:22:27.397 INFO:tasks.rados.rados.0.smithi016.stdout:74: writing smithi01698494-74 from 1622009 to 2348692 tid 2 2023-01-25T05:22:27.398 INFO:tasks.rados.rados.0.smithi016.stdout:74: writing smithi01698494-74 from 3121996 to 3226911 tid 3 2023-01-25T05:22:27.398 INFO:tasks.rados.rados.0.smithi016.stdout:75: write initial oid 75 2023-01-25T05:22:27.398 INFO:tasks.rados.rados.0.smithi016.stdout:75: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2023-01-25T05:22:27.403 INFO:tasks.rados.rados.0.smithi016.stdout:75: writing smithi01698494-75 from 443718 to 1059321 tid 1 2023-01-25T05:22:27.408 INFO:tasks.rados.rados.0.smithi016.stdout:75: writing smithi01698494-75 from 1564138 to 2034479 tid 2 2023-01-25T05:22:27.415 INFO:tasks.rados.rados.0.smithi016.stdout:75: writing smithi01698494-75 from 2633066 to 3243718 tid 3 2023-01-25T05:22:27.415 INFO:tasks.rados.rados.0.smithi016.stdout:76: write initial oid 76 2023-01-25T05:22:27.416 INFO:tasks.rados.rados.0.smithi016.stdout:76: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2023-01-25T05:22:27.422 INFO:tasks.rados.rados.0.smithi016.stdout:76: writing smithi01698494-76 from 460525 to 1151377 tid 1 2023-01-25T05:22:27.428 INFO:tasks.rados.rados.0.smithi016.stdout:76: writing smithi01698494-76 from 1622620 to 2352972 tid 2 2023-01-25T05:22:27.433 INFO:tasks.rados.rados.0.smithi016.stdout:76: writing smithi01698494-76 from 3060489 to 3260525 tid 3 2023-01-25T05:22:27.434 INFO:tasks.rados.rados.0.smithi016.stdout:77: write initial oid 77 2023-01-25T05:22:27.434 INFO:tasks.rados.rados.0.smithi016.stdout:77: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2023-01-25T05:22:27.440 INFO:tasks.rados.rados.0.smithi016.stdout:77: writing smithi01698494-77 from 477332 to 1243433 tid 1 2023-01-25T05:22:27.445 INFO:tasks.rados.rados.0.smithi016.stdout:77: writing smithi01698494-77 from 1681102 to 2155112 tid 2 2023-01-25T05:22:27.452 INFO:tasks.rados.rados.0.smithi016.stdout:77: writing smithi01698494-77 from 2687912 to 3277332 tid 3 2023-01-25T05:22:27.452 INFO:tasks.rados.rados.0.smithi016.stdout:78: write initial oid 78 2023-01-25T05:22:27.452 INFO:tasks.rados.rados.0.smithi016.stdout:78: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2023-01-25T05:22:27.457 INFO:tasks.rados.rados.0.smithi016.stdout:78: writing smithi01698494-78 from 494139 to 1051842 tid 1 2023-01-25T05:22:27.464 INFO:tasks.rados.rados.0.smithi016.stdout:78: writing smithi01698494-78 from 1455937 to 2189958 tid 2 2023-01-25T05:22:27.470 INFO:tasks.rados.rados.0.smithi016.stdout:78: writing smithi01698494-78 from 2948041 to 3294139 tid 3 2023-01-25T05:22:27.470 INFO:tasks.rados.rados.0.smithi016.stdout:79: write initial oid 79 2023-01-25T05:22:27.471 INFO:tasks.rados.rados.0.smithi016.stdout:79: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2023-01-25T05:22:27.477 INFO:tasks.rados.rados.0.smithi016.stdout:79: writing smithi01698494-79 from 510946 to 1143898 tid 1 2023-01-25T05:22:27.483 INFO:tasks.rados.rados.0.smithi016.stdout:79: writing smithi01698494-79 from 1798066 to 2275745 tid 2 2023-01-25T05:22:27.489 INFO:tasks.rados.rados.0.smithi016.stdout:79: writing smithi01698494-79 from 2742758 to 3199975 tid 3 2023-01-25T05:22:27.489 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.489 INFO:tasks.rados.rados.0.smithi016.stdout:63: finishing write tid 4 to smithi01698494-63 2023-01-25T05:22:27.489 INFO:tasks.rados.rados.0.smithi016.stdout:63: oid 63 updating version 0 to 14 2023-01-25T05:22:27.489 INFO:tasks.rados.rados.0.smithi016.stdout:63: oid 63 version 14 is already newer than 13 2023-01-25T05:22:27.490 INFO:tasks.rados.rados.0.smithi016.stdout:63: oid 63 version 14 is already newer than 12 2023-01-25T05:22:27.490 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 63 v 14 (ObjNum 62 snap 0 seq_num 62) dirty exists 2023-01-25T05:22:27.490 INFO:tasks.rados.rados.0.smithi016.stdout:63: left oid 63 (ObjNum 62 snap 0 seq_num 62) 2023-01-25T05:22:27.491 INFO:tasks.rados.rados.0.smithi016.stdout:63: done (15 left) 2023-01-25T05:22:27.491 INFO:tasks.rados.rados.0.smithi016.stdout:80: write initial oid 80 2023-01-25T05:22:27.491 INFO:tasks.rados.rados.0.smithi016.stdout:80: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2023-01-25T05:22:27.495 INFO:tasks.rados.rados.0.smithi016.stdout:80: writing smithi01698494-80 from 527753 to 1235954 tid 1 2023-01-25T05:22:27.502 INFO:tasks.rados.rados.0.smithi016.stdout:80: writing smithi01698494-80 from 1856548 to 2594238 tid 2 2023-01-25T05:22:27.507 INFO:tasks.rados.rados.0.smithi016.stdout:80: writing smithi01698494-80 from 3286534 to 3327753 tid 3 2023-01-25T05:22:27.507 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.508 INFO:tasks.rados.rados.0.smithi016.stdout:65: finishing write tid 1 to smithi01698494-65 2023-01-25T05:22:27.508 INFO:tasks.rados.rados.0.smithi016.stdout:65: finishing write tid 2 to smithi01698494-65 2023-01-25T05:22:27.508 INFO:tasks.rados.rados.0.smithi016.stdout:65: finishing write tid 3 to smithi01698494-65 2023-01-25T05:22:27.508 INFO:tasks.rados.rados.0.smithi016.stdout:65: finishing write tid 4 to smithi01698494-65 2023-01-25T05:22:27.508 INFO:tasks.rados.rados.0.smithi016.stdout:65: oid 65 updating version 0 to 15 2023-01-25T05:22:27.509 INFO:tasks.rados.rados.0.smithi016.stdout:65: oid 65 updating version 15 to 16 2023-01-25T05:22:27.509 INFO:tasks.rados.rados.0.smithi016.stdout:65: oid 65 updating version 16 to 17 2023-01-25T05:22:27.509 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 65 v 17 (ObjNum 64 snap 0 seq_num 64) dirty exists 2023-01-25T05:22:27.509 INFO:tasks.rados.rados.0.smithi016.stdout:65: left oid 65 (ObjNum 64 snap 0 seq_num 64) 2023-01-25T05:22:27.509 INFO:tasks.rados.rados.0.smithi016.stdout:66: finishing write tid 1 to smithi01698494-66 2023-01-25T05:22:27.510 INFO:tasks.rados.rados.0.smithi016.stdout:65: done (15 left) 2023-01-25T05:22:27.510 INFO:tasks.rados.rados.0.smithi016.stdout:81: write initial oid 81 2023-01-25T05:22:27.510 INFO:tasks.rados.rados.0.smithi016.stdout:81: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2023-01-25T05:22:27.513 INFO:tasks.rados.rados.0.smithi016.stdout:81: writing smithi01698494-81 from 544560 to 1328010 tid 1 2023-01-25T05:22:27.520 INFO:tasks.rados.rados.0.smithi016.stdout:81: writing smithi01698494-81 from 1915030 to 2396378 tid 2 2023-01-25T05:22:27.526 INFO:tasks.rados.rados.0.smithi016.stdout:81: writing smithi01698494-81 from 2797604 to 3277365 tid 3 2023-01-25T05:22:27.526 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.526 INFO:tasks.rados.rados.0.smithi016.stdout:66: finishing write tid 2 to smithi01698494-66 2023-01-25T05:22:27.526 INFO:tasks.rados.rados.0.smithi016.stdout:66: finishing write tid 3 to smithi01698494-66 2023-01-25T05:22:27.527 INFO:tasks.rados.rados.0.smithi016.stdout:66: finishing write tid 4 to smithi01698494-66 2023-01-25T05:22:27.527 INFO:tasks.rados.rados.0.smithi016.stdout:66: oid 66 updating version 0 to 37 2023-01-25T05:22:27.527 INFO:tasks.rados.rados.0.smithi016.stdout:66: oid 66 updating version 37 to 38 2023-01-25T05:22:27.527 INFO:tasks.rados.rados.0.smithi016.stdout:66: oid 66 version 38 is already newer than 15 2023-01-25T05:22:27.527 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 66 v 38 (ObjNum 65 snap 0 seq_num 65) dirty exists 2023-01-25T05:22:27.528 INFO:tasks.rados.rados.0.smithi016.stdout:66: left oid 66 (ObjNum 65 snap 0 seq_num 65) 2023-01-25T05:22:27.528 INFO:tasks.rados.rados.0.smithi016.stdout:67: finishing write tid 1 to smithi01698494-67 2023-01-25T05:22:27.528 INFO:tasks.rados.rados.0.smithi016.stdout:66: done (15 left) 2023-01-25T05:22:27.528 INFO:tasks.rados.rados.0.smithi016.stdout:82: write initial oid 82 2023-01-25T05:22:27.528 INFO:tasks.rados.rados.0.smithi016.stdout:82: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2023-01-25T05:22:27.530 INFO:tasks.rados.rados.0.smithi016.stdout:82: writing smithi01698494-82 from 561367 to 1020066 tid 1 2023-01-25T05:22:27.538 INFO:tasks.rados.rados.0.smithi016.stdout:82: writing smithi01698494-82 from 1573512 to 2314871 tid 2 2023-01-25T05:22:27.545 INFO:tasks.rados.rados.0.smithi016.stdout:82: writing smithi01698494-82 from 2941380 to 3361367 tid 3 2023-01-25T05:22:27.545 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.545 INFO:tasks.rados.rados.0.smithi016.stdout:67: finishing write tid 2 to smithi01698494-67 2023-01-25T05:22:27.545 INFO:tasks.rados.rados.0.smithi016.stdout:67: finishing write tid 3 to smithi01698494-67 2023-01-25T05:22:27.545 INFO:tasks.rados.rados.0.smithi016.stdout:67: finishing write tid 4 to smithi01698494-67 2023-01-25T05:22:27.546 INFO:tasks.rados.rados.0.smithi016.stdout:67: finishing write tid 5 to smithi01698494-67 2023-01-25T05:22:27.546 INFO:tasks.rados.rados.0.smithi016.stdout:67: oid 67 updating version 0 to 16 2023-01-25T05:22:27.546 INFO:tasks.rados.rados.0.smithi016.stdout:67: oid 67 updating version 16 to 18 2023-01-25T05:22:27.546 INFO:tasks.rados.rados.0.smithi016.stdout:67: oid 67 updating version 18 to 19 2023-01-25T05:22:27.546 INFO:tasks.rados.rados.0.smithi016.stdout:67: oid 67 version 19 is already newer than 17 2023-01-25T05:22:27.547 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 67 v 19 (ObjNum 66 snap 0 seq_num 66) dirty exists 2023-01-25T05:22:27.547 INFO:tasks.rados.rados.0.smithi016.stdout:67: left oid 67 (ObjNum 66 snap 0 seq_num 66) 2023-01-25T05:22:27.547 INFO:tasks.rados.rados.0.smithi016.stdout:68: finishing write tid 1 to smithi01698494-68 2023-01-25T05:22:27.547 INFO:tasks.rados.rados.0.smithi016.stdout:68: finishing write tid 2 to smithi01698494-68 2023-01-25T05:22:27.547 INFO:tasks.rados.rados.0.smithi016.stdout:68: finishing write tid 3 to smithi01698494-68 2023-01-25T05:22:27.548 INFO:tasks.rados.rados.0.smithi016.stdout:68: finishing write tid 4 to smithi01698494-68 2023-01-25T05:22:27.548 INFO:tasks.rados.rados.0.smithi016.stdout:68: oid 68 updating version 0 to 49 2023-01-25T05:22:27.548 INFO:tasks.rados.rados.0.smithi016.stdout:68: oid 68 updating version 49 to 50 2023-01-25T05:22:27.548 INFO:tasks.rados.rados.0.smithi016.stdout:68: oid 68 version 50 is already newer than 48 2023-01-25T05:22:27.548 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 68 v 50 (ObjNum 67 snap 0 seq_num 67) dirty exists 2023-01-25T05:22:27.549 INFO:tasks.rados.rados.0.smithi016.stdout:68: left oid 68 (ObjNum 67 snap 0 seq_num 67) 2023-01-25T05:22:27.549 INFO:tasks.rados.rados.0.smithi016.stdout:69: finishing write tid 1 to smithi01698494-69 2023-01-25T05:22:27.549 INFO:tasks.rados.rados.0.smithi016.stdout:67: done (15 left) 2023-01-25T05:22:27.549 INFO:tasks.rados.rados.0.smithi016.stdout:68: done (14 left) 2023-01-25T05:22:27.549 INFO:tasks.rados.rados.0.smithi016.stdout:83: write initial oid 83 2023-01-25T05:22:27.550 INFO:tasks.rados.rados.0.smithi016.stdout:83: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2023-01-25T05:22:27.550 INFO:tasks.rados.rados.0.smithi016.stdout:83: writing smithi01698494-83 from 578174 to 1112122 tid 1 2023-01-25T05:22:27.555 INFO:tasks.rados.rados.0.smithi016.stdout:83: writing smithi01698494-83 from 1515641 to 2000658 tid 2 2023-01-25T05:22:27.562 INFO:tasks.rados.rados.0.smithi016.stdout:83: writing smithi01698494-83 from 2736097 to 3238402 tid 3 2023-01-25T05:22:27.562 INFO:tasks.rados.rados.0.smithi016.stdout:84: write initial oid 84 2023-01-25T05:22:27.563 INFO:tasks.rados.rados.0.smithi016.stdout:84: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2023-01-25T05:22:27.568 INFO:tasks.rados.rados.0.smithi016.stdout:84: writing smithi01698494-84 from 594981 to 1204178 tid 1 2023-01-25T05:22:27.577 INFO:tasks.rados.rados.0.smithi016.stdout:84: writing smithi01698494-84 from 1974123 to 2719151 tid 2 2023-01-25T05:22:27.582 INFO:tasks.rados.rados.0.smithi016.stdout:84: writing smithi01698494-84 from 3279873 to 3394981 tid 3 2023-01-25T05:22:27.582 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.582 INFO:tasks.rados.rados.0.smithi016.stdout:69: finishing write tid 2 to smithi01698494-69 2023-01-25T05:22:27.583 INFO:tasks.rados.rados.0.smithi016.stdout:69: finishing write tid 3 to smithi01698494-69 2023-01-25T05:22:27.583 INFO:tasks.rados.rados.0.smithi016.stdout:69: finishing write tid 4 to smithi01698494-69 2023-01-25T05:22:27.583 INFO:tasks.rados.rados.0.smithi016.stdout:69: finishing write tid 5 to smithi01698494-69 2023-01-25T05:22:27.583 INFO:tasks.rados.rados.0.smithi016.stdout:69: oid 69 updating version 0 to 52 2023-01-25T05:22:27.584 INFO:tasks.rados.rados.0.smithi016.stdout:69: oid 69 version 52 is already newer than 51 2023-01-25T05:22:27.584 INFO:tasks.rados.rados.0.smithi016.stdout:69: oid 69 version 52 is already newer than 49 2023-01-25T05:22:27.584 INFO:tasks.rados.rados.0.smithi016.stdout:69: oid 69 updating version 52 to 53 2023-01-25T05:22:27.584 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 69 v 53 (ObjNum 68 snap 0 seq_num 68) dirty exists 2023-01-25T05:22:27.585 INFO:tasks.rados.rados.0.smithi016.stdout:69: left oid 69 (ObjNum 68 snap 0 seq_num 68) 2023-01-25T05:22:27.585 INFO:tasks.rados.rados.0.smithi016.stdout:70: finishing write tid 1 to smithi01698494-70 2023-01-25T05:22:27.585 INFO:tasks.rados.rados.0.smithi016.stdout:70: finishing write tid 2 to smithi01698494-70 2023-01-25T05:22:27.586 INFO:tasks.rados.rados.0.smithi016.stdout:70: finishing write tid 3 to smithi01698494-70 2023-01-25T05:22:27.586 INFO:tasks.rados.rados.0.smithi016.stdout:70: finishing write tid 4 to smithi01698494-70 2023-01-25T05:22:27.586 INFO:tasks.rados.rados.0.smithi016.stdout:69: done (15 left) 2023-01-25T05:22:27.586 INFO:tasks.rados.rados.0.smithi016.stdout:85: write initial oid 85 2023-01-25T05:22:27.587 INFO:tasks.rados.rados.0.smithi016.stdout:85: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2023-01-25T05:22:27.587 INFO:tasks.rados.rados.0.smithi016.stdout:85: writing smithi01698494-85 from 611788 to 1012587 tid 1 2023-01-25T05:22:27.593 INFO:tasks.rados.rados.0.smithi016.stdout:85: writing smithi01698494-85 from 1748958 to 2237644 tid 2 2023-01-25T05:22:27.600 INFO:tasks.rados.rados.0.smithi016.stdout:85: writing smithi01698494-85 from 2907296 to 3411788 tid 3 2023-01-25T05:22:27.600 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.600 INFO:tasks.rados.rados.0.smithi016.stdout:70: finishing write tid 5 to smithi01698494-70 2023-01-25T05:22:27.600 INFO:tasks.rados.rados.0.smithi016.stdout:70: oid 70 updating version 0 to 20 2023-01-25T05:22:27.601 INFO:tasks.rados.rados.0.smithi016.stdout:70: oid 70 updating version 20 to 23 2023-01-25T05:22:27.601 INFO:tasks.rados.rados.0.smithi016.stdout:70: oid 70 version 23 is already newer than 21 2023-01-25T05:22:27.601 INFO:tasks.rados.rados.0.smithi016.stdout:70: oid 70 version 23 is already newer than 22 2023-01-25T05:22:27.601 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 70 v 23 (ObjNum 69 snap 0 seq_num 69) dirty exists 2023-01-25T05:22:27.601 INFO:tasks.rados.rados.0.smithi016.stdout:70: left oid 70 (ObjNum 69 snap 0 seq_num 69) 2023-01-25T05:22:27.602 INFO:tasks.rados.rados.0.smithi016.stdout:71: finishing write tid 1 to smithi01698494-71 2023-01-25T05:22:27.602 INFO:tasks.rados.rados.0.smithi016.stdout:71: finishing write tid 2 to smithi01698494-71 2023-01-25T05:22:27.602 INFO:tasks.rados.rados.0.smithi016.stdout:71: finishing write tid 3 to smithi01698494-71 2023-01-25T05:22:27.602 INFO:tasks.rados.rados.0.smithi016.stdout:70: done (15 left) 2023-01-25T05:22:27.603 INFO:tasks.rados.rados.0.smithi016.stdout:86: write initial oid 86 2023-01-25T05:22:27.603 INFO:tasks.rados.rados.0.smithi016.stdout:86: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2023-01-25T05:22:27.605 INFO:tasks.rados.rados.0.smithi016.stdout:86: writing smithi01698494-86 from 628595 to 1104643 tid 1 2023-01-25T05:22:27.613 INFO:tasks.rados.rados.0.smithi016.stdout:86: writing smithi01698494-86 from 1807440 to 2439784 tid 2 2023-01-25T05:22:27.618 INFO:tasks.rados.rados.0.smithi016.stdout:86: writing smithi01698494-86 from 2934719 to 3387193 tid 3 2023-01-25T05:22:27.619 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.619 INFO:tasks.rados.rados.0.smithi016.stdout:71: finishing write tid 4 to smithi01698494-71 2023-01-25T05:22:27.619 INFO:tasks.rados.rados.0.smithi016.stdout:71: oid 71 updating version 0 to 20 2023-01-25T05:22:27.620 INFO:tasks.rados.rados.0.smithi016.stdout:71: oid 71 version 20 is already newer than 18 2023-01-25T05:22:27.620 INFO:tasks.rados.rados.0.smithi016.stdout:71: oid 71 version 20 is already newer than 19 2023-01-25T05:22:27.620 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 71 v 20 (ObjNum 70 snap 0 seq_num 70) dirty exists 2023-01-25T05:22:27.620 INFO:tasks.rados.rados.0.smithi016.stdout:71: left oid 71 (ObjNum 70 snap 0 seq_num 70) 2023-01-25T05:22:27.620 INFO:tasks.rados.rados.0.smithi016.stdout:72: finishing write tid 1 to smithi01698494-72 2023-01-25T05:22:27.621 INFO:tasks.rados.rados.0.smithi016.stdout:71: done (15 left) 2023-01-25T05:22:27.621 INFO:tasks.rados.rados.0.smithi016.stdout:87: write initial oid 87 2023-01-25T05:22:27.621 INFO:tasks.rados.rados.0.smithi016.stdout:87: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2023-01-25T05:22:27.624 INFO:tasks.rados.rados.0.smithi016.stdout:87: writing smithi01698494-87 from 645402 to 1196699 tid 1 2023-01-25T05:22:27.630 INFO:tasks.rados.rados.0.smithi016.stdout:87: writing smithi01698494-87 from 1749569 to 2241924 tid 2 2023-01-25T05:22:27.638 INFO:tasks.rados.rados.0.smithi016.stdout:87: writing smithi01698494-87 from 2845789 to 3445402 tid 3 2023-01-25T05:22:27.638 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.638 INFO:tasks.rados.rados.0.smithi016.stdout:72: finishing write tid 2 to smithi01698494-72 2023-01-25T05:22:27.638 INFO:tasks.rados.rados.0.smithi016.stdout:72: finishing write tid 3 to smithi01698494-72 2023-01-25T05:22:27.638 INFO:tasks.rados.rados.0.smithi016.stdout:72: finishing write tid 4 to smithi01698494-72 2023-01-25T05:22:27.639 INFO:tasks.rados.rados.0.smithi016.stdout:72: oid 72 updating version 0 to 30 2023-01-25T05:22:27.639 INFO:tasks.rados.rados.0.smithi016.stdout:72: oid 72 updating version 30 to 32 2023-01-25T05:22:27.639 INFO:tasks.rados.rados.0.smithi016.stdout:72: oid 72 version 32 is already newer than 31 2023-01-25T05:22:27.639 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 72 v 32 (ObjNum 71 snap 0 seq_num 71) dirty exists 2023-01-25T05:22:27.639 INFO:tasks.rados.rados.0.smithi016.stdout:72: left oid 72 (ObjNum 71 snap 0 seq_num 71) 2023-01-25T05:22:27.640 INFO:tasks.rados.rados.0.smithi016.stdout:73: finishing write tid 1 to smithi01698494-73 2023-01-25T05:22:27.640 INFO:tasks.rados.rados.0.smithi016.stdout:73: finishing write tid 2 to smithi01698494-73 2023-01-25T05:22:27.640 INFO:tasks.rados.rados.0.smithi016.stdout:73: finishing write tid 3 to smithi01698494-73 2023-01-25T05:22:27.640 INFO:tasks.rados.rados.0.smithi016.stdout:72: done (15 left) 2023-01-25T05:22:27.640 INFO:tasks.rados.rados.0.smithi016.stdout:88: write initial oid 88 2023-01-25T05:22:27.641 INFO:tasks.rados.rados.0.smithi016.stdout:88: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2023-01-25T05:22:27.644 INFO:tasks.rados.rados.0.smithi016.stdout:88: writing smithi01698494-88 from 662209 to 1288755 tid 1 2023-01-25T05:22:27.650 INFO:tasks.rados.rados.0.smithi016.stdout:88: writing smithi01698494-88 from 1808051 to 2444064 tid 2 2023-01-25T05:22:27.656 INFO:tasks.rados.rados.0.smithi016.stdout:88: writing smithi01698494-88 from 2873212 to 3462209 tid 3 2023-01-25T05:22:27.656 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.657 INFO:tasks.rados.rados.0.smithi016.stdout:73: finishing write tid 4 to smithi01698494-73 2023-01-25T05:22:27.657 INFO:tasks.rados.rados.0.smithi016.stdout:73: finishing write tid 5 to smithi01698494-73 2023-01-25T05:22:27.657 INFO:tasks.rados.rados.0.smithi016.stdout:73: oid 73 updating version 0 to 22 2023-01-25T05:22:27.657 INFO:tasks.rados.rados.0.smithi016.stdout:73: oid 73 updating version 22 to 23 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:73: oid 73 updating version 23 to 24 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:73: oid 73 version 24 is already newer than 21 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 73 v 24 (ObjNum 72 snap 0 seq_num 72) dirty exists 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:73: left oid 73 (ObjNum 72 snap 0 seq_num 72) 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:74: finishing write tid 1 to smithi01698494-74 2023-01-25T05:22:27.658 INFO:tasks.rados.rados.0.smithi016.stdout:74: finishing write tid 2 to smithi01698494-74 2023-01-25T05:22:27.659 INFO:tasks.rados.rados.0.smithi016.stdout:74: finishing write tid 3 to smithi01698494-74 2023-01-25T05:22:27.659 INFO:tasks.rados.rados.0.smithi016.stdout:74: finishing write tid 4 to smithi01698494-74 2023-01-25T05:22:27.659 INFO:tasks.rados.rados.0.smithi016.stdout:74: finishing write tid 5 to smithi01698494-74 2023-01-25T05:22:27.659 INFO:tasks.rados.rados.0.smithi016.stdout:74: oid 74 updating version 0 to 28 2023-01-25T05:22:27.659 INFO:tasks.rados.rados.0.smithi016.stdout:74: oid 74 version 28 is already newer than 26 2023-01-25T05:22:27.660 INFO:tasks.rados.rados.0.smithi016.stdout:74: oid 74 version 28 is already newer than 25 2023-01-25T05:22:27.660 INFO:tasks.rados.rados.0.smithi016.stdout:74: oid 74 version 28 is already newer than 27 2023-01-25T05:22:27.660 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 74 v 28 (ObjNum 73 snap 0 seq_num 73) dirty exists 2023-01-25T05:22:27.660 INFO:tasks.rados.rados.0.smithi016.stdout:74: left oid 74 (ObjNum 73 snap 0 seq_num 73) 2023-01-25T05:22:27.660 INFO:tasks.rados.rados.0.smithi016.stdout:75: finishing write tid 1 to smithi01698494-75 2023-01-25T05:22:27.661 INFO:tasks.rados.rados.0.smithi016.stdout:75: finishing write tid 2 to smithi01698494-75 2023-01-25T05:22:27.661 INFO:tasks.rados.rados.0.smithi016.stdout:73: done (15 left) 2023-01-25T05:22:27.661 INFO:tasks.rados.rados.0.smithi016.stdout:74: done (14 left) 2023-01-25T05:22:27.661 INFO:tasks.rados.rados.0.smithi016.stdout:89: write initial oid 89 2023-01-25T05:22:27.661 INFO:tasks.rados.rados.0.smithi016.stdout:89: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2023-01-25T05:22:27.663 INFO:tasks.rados.rados.0.smithi016.stdout:89: writing smithi01698494-89 from 679016 to 1380811 tid 1 2023-01-25T05:22:27.669 INFO:tasks.rados.rados.0.smithi016.stdout:89: writing smithi01698494-89 from 1866533 to 2362557 tid 2 2023-01-25T05:22:27.674 INFO:tasks.rados.rados.0.smithi016.stdout:89: writing smithi01698494-89 from 2900635 to 3303278 tid 3 2023-01-25T05:22:27.674 INFO:tasks.rados.rados.0.smithi016.stdout:90: write initial oid 90 2023-01-25T05:22:27.674 INFO:tasks.rados.rados.0.smithi016.stdout:90: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2023-01-25T05:22:27.682 INFO:tasks.rados.rados.0.smithi016.stdout:90: writing smithi01698494-90 from 695823 to 1472867 tid 1 2023-01-25T05:22:27.688 INFO:tasks.rados.rados.0.smithi016.stdout:90: writing smithi01698494-90 from 1925015 to 2564697 tid 2 2023-01-25T05:22:27.694 INFO:tasks.rados.rados.0.smithi016.stdout:90: writing smithi01698494-90 from 3328058 to 3495823 tid 3 2023-01-25T05:22:27.694 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.694 INFO:tasks.rados.rados.0.smithi016.stdout:75: finishing write tid 3 to smithi01698494-75 2023-01-25T05:22:27.694 INFO:tasks.rados.rados.0.smithi016.stdout:75: finishing write tid 4 to smithi01698494-75 2023-01-25T05:22:27.694 INFO:tasks.rados.rados.0.smithi016.stdout:75: finishing write tid 5 to smithi01698494-75 2023-01-25T05:22:27.695 INFO:tasks.rados.rados.0.smithi016.stdout:75: oid 75 updating version 0 to 27 2023-01-25T05:22:27.695 INFO:tasks.rados.rados.0.smithi016.stdout:75: oid 75 version 27 is already newer than 25 2023-01-25T05:22:27.696 INFO:tasks.rados.rados.0.smithi016.stdout:75: oid 75 version 27 is already newer than 26 2023-01-25T05:22:27.696 INFO:tasks.rados.rados.0.smithi016.stdout:75: oid 75 version 27 is already newer than 24 2023-01-25T05:22:27.696 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 75 v 27 (ObjNum 74 snap 0 seq_num 74) dirty exists 2023-01-25T05:22:27.696 INFO:tasks.rados.rados.0.smithi016.stdout:75: left oid 75 (ObjNum 74 snap 0 seq_num 74) 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: finishing write tid 1 to smithi01698494-76 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: finishing write tid 2 to smithi01698494-76 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: finishing write tid 3 to smithi01698494-76 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: finishing write tid 4 to smithi01698494-76 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: finishing write tid 5 to smithi01698494-76 2023-01-25T05:22:27.697 INFO:tasks.rados.rados.0.smithi016.stdout:76: oid 76 updating version 0 to 54 2023-01-25T05:22:27.698 INFO:tasks.rados.rados.0.smithi016.stdout:76: oid 76 version 54 is already newer than 50 2023-01-25T05:22:27.698 INFO:tasks.rados.rados.0.smithi016.stdout:76: oid 76 updating version 54 to 55 2023-01-25T05:22:27.698 INFO:tasks.rados.rados.0.smithi016.stdout:76: oid 76 updating version 55 to 56 2023-01-25T05:22:27.699 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 76 v 56 (ObjNum 75 snap 0 seq_num 75) dirty exists 2023-01-25T05:22:27.699 INFO:tasks.rados.rados.0.smithi016.stdout:76: left oid 76 (ObjNum 75 snap 0 seq_num 75) 2023-01-25T05:22:27.699 INFO:tasks.rados.rados.0.smithi016.stdout:75: done (15 left) 2023-01-25T05:22:27.699 INFO:tasks.rados.rados.0.smithi016.stdout:76: done (14 left) 2023-01-25T05:22:27.699 INFO:tasks.rados.rados.0.smithi016.stdout:91: write initial oid 91 2023-01-25T05:22:27.700 INFO:tasks.rados.rados.0.smithi016.stdout:91: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2023-01-25T05:22:27.700 INFO:tasks.rados.rados.0.smithi016.stdout:91: writing smithi01698494-91 from 712630 to 1164923 tid 1 2023-01-25T05:22:27.704 INFO:tasks.rados.rados.0.smithi016.stdout:91: writing smithi01698494-91 from 1583497 to 2083190 tid 2 2023-01-25T05:22:27.709 INFO:tasks.rados.rados.0.smithi016.stdout:91: writing smithi01698494-91 from 2555481 to 2980668 tid 3 2023-01-25T05:22:27.710 INFO:tasks.rados.rados.0.smithi016.stdout:92: write initial oid 92 2023-01-25T05:22:27.710 INFO:tasks.rados.rados.0.smithi016.stdout:92: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2023-01-25T05:22:27.718 INFO:tasks.rados.rados.0.smithi016.stdout:92: writing smithi01698494-92 from 729437 to 1256979 tid 1 2023-01-25T05:22:27.724 INFO:tasks.rados.rados.0.smithi016.stdout:92: writing smithi01698494-92 from 1925626 to 2568977 tid 2 2023-01-25T05:22:27.730 INFO:tasks.rados.rados.0.smithi016.stdout:92: writing smithi01698494-92 from 3266551 to 3529437 tid 3 2023-01-25T05:22:27.730 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.730 INFO:tasks.rados.rados.0.smithi016.stdout:77: finishing write tid 1 to smithi01698494-77 2023-01-25T05:22:27.731 INFO:tasks.rados.rados.0.smithi016.stdout:77: finishing write tid 2 to smithi01698494-77 2023-01-25T05:22:27.731 INFO:tasks.rados.rados.0.smithi016.stdout:77: finishing write tid 3 to smithi01698494-77 2023-01-25T05:22:27.731 INFO:tasks.rados.rados.0.smithi016.stdout:77: finishing write tid 4 to smithi01698494-77 2023-01-25T05:22:27.731 INFO:tasks.rados.rados.0.smithi016.stdout:77: finishing write tid 5 to smithi01698494-77 2023-01-25T05:22:27.731 INFO:tasks.rados.rados.0.smithi016.stdout:77: oid 77 updating version 0 to 29 2023-01-25T05:22:27.732 INFO:tasks.rados.rados.0.smithi016.stdout:77: oid 77 updating version 29 to 31 2023-01-25T05:22:27.732 INFO:tasks.rados.rados.0.smithi016.stdout:77: oid 77 version 31 is already newer than 30 2023-01-25T05:22:27.732 INFO:tasks.rados.rados.0.smithi016.stdout:77: oid 77 version 31 is already newer than 28 2023-01-25T05:22:27.732 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 77 v 31 (ObjNum 76 snap 0 seq_num 76) dirty exists 2023-01-25T05:22:27.732 INFO:tasks.rados.rados.0.smithi016.stdout:77: left oid 77 (ObjNum 76 snap 0 seq_num 76) 2023-01-25T05:22:27.733 INFO:tasks.rados.rados.0.smithi016.stdout:78: finishing write tid 1 to smithi01698494-78 2023-01-25T05:22:27.733 INFO:tasks.rados.rados.0.smithi016.stdout:78: finishing write tid 2 to smithi01698494-78 2023-01-25T05:22:27.733 INFO:tasks.rados.rados.0.smithi016.stdout:78: finishing write tid 3 to smithi01698494-78 2023-01-25T05:22:27.733 INFO:tasks.rados.rados.0.smithi016.stdout:78: finishing write tid 4 to smithi01698494-78 2023-01-25T05:22:27.733 INFO:tasks.rados.rados.0.smithi016.stdout:78: finishing write tid 5 to smithi01698494-78 2023-01-25T05:22:27.734 INFO:tasks.rados.rados.0.smithi016.stdout:78: oid 78 updating version 0 to 31 2023-01-25T05:22:27.734 INFO:tasks.rados.rados.0.smithi016.stdout:78: oid 78 version 31 is already newer than 29 2023-01-25T05:22:27.734 INFO:tasks.rados.rados.0.smithi016.stdout:78: oid 78 version 31 is already newer than 30 2023-01-25T05:22:27.734 INFO:tasks.rados.rados.0.smithi016.stdout:78: oid 78 updating version 31 to 32 2023-01-25T05:22:27.734 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 78 v 32 (ObjNum 77 snap 0 seq_num 77) dirty exists 2023-01-25T05:22:27.735 INFO:tasks.rados.rados.0.smithi016.stdout:78: left oid 78 (ObjNum 77 snap 0 seq_num 77) 2023-01-25T05:22:27.735 INFO:tasks.rados.rados.0.smithi016.stdout:79: finishing write tid 1 to smithi01698494-79 2023-01-25T05:22:27.735 INFO:tasks.rados.rados.0.smithi016.stdout:79: finishing write tid 2 to smithi01698494-79 2023-01-25T05:22:27.735 INFO:tasks.rados.rados.0.smithi016.stdout:79: finishing write tid 3 to smithi01698494-79 2023-01-25T05:22:27.735 INFO:tasks.rados.rados.0.smithi016.stdout:79: finishing write tid 4 to smithi01698494-79 2023-01-25T05:22:27.736 INFO:tasks.rados.rados.0.smithi016.stdout:79: finishing write tid 5 to smithi01698494-79 2023-01-25T05:22:27.736 INFO:tasks.rados.rados.0.smithi016.stdout:79: oid 79 updating version 0 to 36 2023-01-25T05:22:27.736 INFO:tasks.rados.rados.0.smithi016.stdout:79: oid 79 version 36 is already newer than 33 2023-01-25T05:22:27.736 INFO:tasks.rados.rados.0.smithi016.stdout:79: oid 79 version 36 is already newer than 34 2023-01-25T05:22:27.736 INFO:tasks.rados.rados.0.smithi016.stdout:79: oid 79 version 36 is already newer than 35 2023-01-25T05:22:27.737 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 79 v 36 (ObjNum 78 snap 0 seq_num 78) dirty exists 2023-01-25T05:22:27.737 INFO:tasks.rados.rados.0.smithi016.stdout:79: left oid 79 (ObjNum 78 snap 0 seq_num 78) 2023-01-25T05:22:27.737 INFO:tasks.rados.rados.0.smithi016.stdout:80: finishing write tid 1 to smithi01698494-80 2023-01-25T05:22:27.737 INFO:tasks.rados.rados.0.smithi016.stdout:77: done (15 left) 2023-01-25T05:22:27.737 INFO:tasks.rados.rados.0.smithi016.stdout:78: done (14 left) 2023-01-25T05:22:27.738 INFO:tasks.rados.rados.0.smithi016.stdout:79: done (13 left) 2023-01-25T05:22:27.738 INFO:tasks.rados.rados.0.smithi016.stdout:93: write initial oid 93 2023-01-25T05:22:27.738 INFO:tasks.rados.rados.0.smithi016.stdout:93: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2023-01-25T05:22:27.739 INFO:tasks.rados.rados.0.smithi016.stdout:93: writing smithi01698494-93 from 746244 to 1465388 tid 1 2023-01-25T05:22:27.743 INFO:tasks.rados.rados.0.smithi016.stdout:93: writing smithi01698494-93 from 2100461 to 2603823 tid 2 2023-01-25T05:22:27.748 INFO:tasks.rados.rados.0.smithi016.stdout:93: writing smithi01698494-93 from 3126680 to 3546244 tid 3 2023-01-25T05:22:27.749 INFO:tasks.rados.rados.0.smithi016.stdout:94: write initial oid 94 2023-01-25T05:22:27.749 INFO:tasks.rados.rados.0.smithi016.stdout:94: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2023-01-25T05:22:27.757 INFO:tasks.rados.rados.0.smithi016.stdout:94: writing smithi01698494-94 from 763051 to 1557444 tid 1 2023-01-25T05:22:27.764 INFO:tasks.rados.rados.0.smithi016.stdout:94: writing smithi01698494-94 from 2158943 to 2805963 tid 2 2023-01-25T05:22:27.768 INFO:tasks.rados.rados.0.smithi016.stdout:94: writing smithi01698494-94 from 3437750 to 3563051 tid 3 2023-01-25T05:22:27.768 INFO:tasks.rados.rados.0.smithi016.stdout:95: write initial oid 95 2023-01-25T05:22:27.769 INFO:tasks.rados.rados.0.smithi016.stdout:95: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2023-01-25T05:22:27.775 INFO:tasks.rados.rados.0.smithi016.stdout:95: writing smithi01698494-95 from 779858 to 1249500 tid 1 2023-01-25T05:22:27.781 INFO:tasks.rados.rados.0.smithi016.stdout:95: writing smithi01698494-95 from 1817425 to 2324456 tid 2 2023-01-25T05:22:27.788 INFO:tasks.rados.rados.0.smithi016.stdout:95: writing smithi01698494-95 from 2781526 to 3368154 tid 3 2023-01-25T05:22:27.788 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.789 INFO:tasks.rados.rados.0.smithi016.stdout:80: finishing write tid 2 to smithi01698494-80 2023-01-25T05:22:27.789 INFO:tasks.rados.rados.0.smithi016.stdout:80: finishing write tid 3 to smithi01698494-80 2023-01-25T05:22:27.789 INFO:tasks.rados.rados.0.smithi016.stdout:80: finishing write tid 4 to smithi01698494-80 2023-01-25T05:22:27.789 INFO:tasks.rados.rados.0.smithi016.stdout:80: finishing write tid 5 to smithi01698494-80 2023-01-25T05:22:27.789 INFO:tasks.rados.rados.0.smithi016.stdout:80: oid 80 updating version 0 to 59 2023-01-25T05:22:27.790 INFO:tasks.rados.rados.0.smithi016.stdout:80: oid 80 version 59 is already newer than 51 2023-01-25T05:22:27.790 INFO:tasks.rados.rados.0.smithi016.stdout:80: oid 80 version 59 is already newer than 58 2023-01-25T05:22:27.790 INFO:tasks.rados.rados.0.smithi016.stdout:80: oid 80 version 59 is already newer than 57 2023-01-25T05:22:27.790 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 80 v 59 (ObjNum 79 snap 0 seq_num 79) dirty exists 2023-01-25T05:22:27.790 INFO:tasks.rados.rados.0.smithi016.stdout:80: left oid 80 (ObjNum 79 snap 0 seq_num 79) 2023-01-25T05:22:27.791 INFO:tasks.rados.rados.0.smithi016.stdout:81: finishing write tid 1 to smithi01698494-81 2023-01-25T05:22:27.791 INFO:tasks.rados.rados.0.smithi016.stdout:81: finishing write tid 2 to smithi01698494-81 2023-01-25T05:22:27.791 INFO:tasks.rados.rados.0.smithi016.stdout:81: finishing write tid 3 to smithi01698494-81 2023-01-25T05:22:27.791 INFO:tasks.rados.rados.0.smithi016.stdout:81: finishing write tid 4 to smithi01698494-81 2023-01-25T05:22:27.791 INFO:tasks.rados.rados.0.smithi016.stdout:81: finishing write tid 5 to smithi01698494-81 2023-01-25T05:22:27.792 INFO:tasks.rados.rados.0.smithi016.stdout:81: oid 81 updating version 0 to 33 2023-01-25T05:22:27.792 INFO:tasks.rados.rados.0.smithi016.stdout:81: oid 81 updating version 33 to 34 2023-01-25T05:22:27.792 INFO:tasks.rados.rados.0.smithi016.stdout:81: oid 81 updating version 34 to 35 2023-01-25T05:22:27.792 INFO:tasks.rados.rados.0.smithi016.stdout:81: oid 81 version 35 is already newer than 32 2023-01-25T05:22:27.792 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 81 v 35 (ObjNum 80 snap 0 seq_num 80) dirty exists 2023-01-25T05:22:27.793 INFO:tasks.rados.rados.0.smithi016.stdout:81: left oid 81 (ObjNum 80 snap 0 seq_num 80) 2023-01-25T05:22:27.793 INFO:tasks.rados.rados.0.smithi016.stdout:82: finishing write tid 1 to smithi01698494-82 2023-01-25T05:22:27.793 INFO:tasks.rados.rados.0.smithi016.stdout:82: finishing write tid 2 to smithi01698494-82 2023-01-25T05:22:27.793 INFO:tasks.rados.rados.0.smithi016.stdout:82: finishing write tid 3 to smithi01698494-82 2023-01-25T05:22:27.793 INFO:tasks.rados.rados.0.smithi016.stdout:82: finishing write tid 4 to smithi01698494-82 2023-01-25T05:22:27.794 INFO:tasks.rados.rados.0.smithi016.stdout:82: finishing write tid 5 to smithi01698494-82 2023-01-25T05:22:27.794 INFO:tasks.rados.rados.0.smithi016.stdout:82: oid 82 updating version 0 to 35 2023-01-25T05:22:27.794 INFO:tasks.rados.rados.0.smithi016.stdout:82: oid 82 version 35 is already newer than 34 2023-01-25T05:22:27.794 INFO:tasks.rados.rados.0.smithi016.stdout:82: oid 82 updating version 35 to 36 2023-01-25T05:22:27.794 INFO:tasks.rados.rados.0.smithi016.stdout:82: oid 82 version 36 is already newer than 33 2023-01-25T05:22:27.795 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 82 v 36 (ObjNum 81 snap 0 seq_num 81) dirty exists 2023-01-25T05:22:27.795 INFO:tasks.rados.rados.0.smithi016.stdout:82: left oid 82 (ObjNum 81 snap 0 seq_num 81) 2023-01-25T05:22:27.795 INFO:tasks.rados.rados.0.smithi016.stdout:80: done (15 left) 2023-01-25T05:22:27.795 INFO:tasks.rados.rados.0.smithi016.stdout:81: done (14 left) 2023-01-25T05:22:27.796 INFO:tasks.rados.rados.0.smithi016.stdout:82: done (13 left) 2023-01-25T05:22:27.796 INFO:tasks.rados.rados.0.smithi016.stdout:96: write initial oid 96 2023-01-25T05:22:27.796 INFO:tasks.rados.rados.0.smithi016.stdout:96: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2023-01-25T05:22:27.796 INFO:tasks.rados.rados.0.smithi016.stdout:96: writing smithi01698494-96 from 796665 to 1341556 tid 1 2023-01-25T05:22:27.800 INFO:tasks.rados.rados.0.smithi016.stdout:96: writing smithi01698494-96 from 1759554 to 2410243 tid 2 2023-01-25T05:22:27.806 INFO:tasks.rados.rados.0.smithi016.stdout:96: writing smithi01698494-96 from 2976243 to 3596665 tid 3 2023-01-25T05:22:27.806 INFO:tasks.rados.rados.0.smithi016.stdout:97: write initial oid 97 2023-01-25T05:22:27.807 INFO:tasks.rados.rados.0.smithi016.stdout:97: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2023-01-25T05:22:27.812 INFO:tasks.rados.rados.0.smithi016.stdout:97: writing smithi01698494-97 from 413472 to 1033612 tid 1 2023-01-25T05:22:27.819 INFO:tasks.rados.rados.0.smithi016.stdout:97: writing smithi01698494-97 from 1818036 to 2328736 tid 2 2023-01-25T05:22:27.826 INFO:tasks.rados.rados.0.smithi016.stdout:97: writing smithi01698494-97 from 3120019 to 3613472 tid 3 2023-01-25T05:22:27.826 INFO:tasks.rados.rados.0.smithi016.stdout:98: write initial oid 98 2023-01-25T05:22:27.826 INFO:tasks.rados.rados.0.smithi016.stdout:98: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2023-01-25T05:22:27.832 INFO:tasks.rados.rados.0.smithi016.stdout:98: writing smithi01698494-98 from 430279 to 1125668 tid 1 2023-01-25T05:22:27.840 INFO:tasks.rados.rados.0.smithi016.stdout:98: writing smithi01698494-98 from 1876518 to 2530876 tid 2 2023-01-25T05:22:27.846 INFO:tasks.rados.rados.0.smithi016.stdout:98: writing smithi01698494-98 from 3031089 to 3567886 tid 3 2023-01-25T05:22:27.846 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.847 INFO:tasks.rados.rados.0.smithi016.stdout:83: finishing write tid 1 to smithi01698494-83 2023-01-25T05:22:27.847 INFO:tasks.rados.rados.0.smithi016.stdout:83: finishing write tid 2 to smithi01698494-83 2023-01-25T05:22:27.847 INFO:tasks.rados.rados.0.smithi016.stdout:83: finishing write tid 3 to smithi01698494-83 2023-01-25T05:22:27.847 INFO:tasks.rados.rados.0.smithi016.stdout:83: finishing write tid 4 to smithi01698494-83 2023-01-25T05:22:27.847 INFO:tasks.rados.rados.0.smithi016.stdout:83: finishing write tid 5 to smithi01698494-83 2023-01-25T05:22:27.848 INFO:tasks.rados.rados.0.smithi016.stdout:83: oid 83 updating version 0 to 40 2023-01-25T05:22:27.848 INFO:tasks.rados.rados.0.smithi016.stdout:83: oid 83 version 40 is already newer than 39 2023-01-25T05:22:27.848 INFO:tasks.rados.rados.0.smithi016.stdout:83: oid 83 version 40 is already newer than 37 2023-01-25T05:22:27.848 INFO:tasks.rados.rados.0.smithi016.stdout:83: oid 83 version 40 is already newer than 38 2023-01-25T05:22:27.849 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 83 v 40 (ObjNum 82 snap 0 seq_num 82) dirty exists 2023-01-25T05:22:27.849 INFO:tasks.rados.rados.0.smithi016.stdout:83: left oid 83 (ObjNum 82 snap 0 seq_num 82) 2023-01-25T05:22:27.849 INFO:tasks.rados.rados.0.smithi016.stdout:84: finishing write tid 1 to smithi01698494-84 2023-01-25T05:22:27.849 INFO:tasks.rados.rados.0.smithi016.stdout:84: finishing write tid 2 to smithi01698494-84 2023-01-25T05:22:27.849 INFO:tasks.rados.rados.0.smithi016.stdout:84: finishing write tid 3 to smithi01698494-84 2023-01-25T05:22:27.850 INFO:tasks.rados.rados.0.smithi016.stdout:84: finishing write tid 4 to smithi01698494-84 2023-01-25T05:22:27.850 INFO:tasks.rados.rados.0.smithi016.stdout:84: finishing write tid 5 to smithi01698494-84 2023-01-25T05:22:27.850 INFO:tasks.rados.rados.0.smithi016.stdout:84: oid 84 updating version 0 to 39 2023-01-25T05:22:27.850 INFO:tasks.rados.rados.0.smithi016.stdout:84: oid 84 version 39 is already newer than 38 2023-01-25T05:22:27.850 INFO:tasks.rados.rados.0.smithi016.stdout:84: oid 84 version 39 is already newer than 37 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:84: oid 84 version 39 is already newer than 36 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 84 v 39 (ObjNum 83 snap 0 seq_num 83) dirty exists 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:84: left oid 84 (ObjNum 83 snap 0 seq_num 83) 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:85: finishing write tid 1 to smithi01698494-85 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:85: finishing write tid 2 to smithi01698494-85 2023-01-25T05:22:27.851 INFO:tasks.rados.rados.0.smithi016.stdout:85: finishing write tid 3 to smithi01698494-85 2023-01-25T05:22:27.852 INFO:tasks.rados.rados.0.smithi016.stdout:85: finishing write tid 4 to smithi01698494-85 2023-01-25T05:22:27.852 INFO:tasks.rados.rados.0.smithi016.stdout:85: finishing write tid 5 to smithi01698494-85 2023-01-25T05:22:27.852 INFO:tasks.rados.rados.0.smithi016.stdout:85: oid 85 updating version 0 to 39 2023-01-25T05:22:27.852 INFO:tasks.rados.rados.0.smithi016.stdout:85: oid 85 version 39 is already newer than 37 2023-01-25T05:22:27.852 INFO:tasks.rados.rados.0.smithi016.stdout:85: oid 85 version 39 is already newer than 38 2023-01-25T05:22:27.853 INFO:tasks.rados.rados.0.smithi016.stdout:85: oid 85 updating version 39 to 40 2023-01-25T05:22:27.853 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 85 v 40 (ObjNum 84 snap 0 seq_num 84) dirty exists 2023-01-25T05:22:27.853 INFO:tasks.rados.rados.0.smithi016.stdout:85: left oid 85 (ObjNum 84 snap 0 seq_num 84) 2023-01-25T05:22:27.853 INFO:tasks.rados.rados.0.smithi016.stdout:83: done (15 left) 2023-01-25T05:22:27.853 INFO:tasks.rados.rados.0.smithi016.stdout:84: done (14 left) 2023-01-25T05:22:27.854 INFO:tasks.rados.rados.0.smithi016.stdout:85: done (13 left) 2023-01-25T05:22:27.854 INFO:tasks.rados.rados.0.smithi016.stdout:99: write initial oid 99 2023-01-25T05:22:27.854 INFO:tasks.rados.rados.0.smithi016.stdout:99: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2023-01-25T05:22:27.854 INFO:tasks.rados.rados.0.smithi016.stdout:99: writing smithi01698494-99 from 447086 to 1217724 tid 1 2023-01-25T05:22:27.860 INFO:tasks.rados.rados.0.smithi016.stdout:99: writing smithi01698494-99 from 1935000 to 2733016 tid 2 2023-01-25T05:22:27.866 INFO:tasks.rados.rados.0.smithi016.stdout:99: writing smithi01698494-99 from 3458512 to 3647086 tid 3 2023-01-25T05:22:27.866 INFO:tasks.rados.rados.0.smithi016.stdout:100: write initial oid 100 2023-01-25T05:22:27.866 INFO:tasks.rados.rados.0.smithi016.stdout:100: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2023-01-25T05:22:27.871 INFO:tasks.rados.rados.0.smithi016.stdout:100: writing smithi01698494-100 from 463893 to 1026133 tid 1 2023-01-25T05:22:27.878 INFO:tasks.rados.rados.0.smithi016.stdout:100: writing smithi01698494-100 from 1593482 to 2251509 tid 2 2023-01-25T05:22:27.883 INFO:tasks.rados.rados.0.smithi016.stdout:100: writing smithi01698494-100 from 2685935 to 3245276 tid 3 2023-01-25T05:22:27.883 INFO:tasks.rados.rados.0.smithi016.stdout:101: write initial oid 101 2023-01-25T05:22:27.884 INFO:tasks.rados.rados.0.smithi016.stdout:101: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2023-01-25T05:22:27.889 INFO:tasks.rados.rados.0.smithi016.stdout:101: writing smithi01698494-101 from 480700 to 1118189 tid 1 2023-01-25T05:22:27.894 INFO:tasks.rados.rados.0.smithi016.stdout:101: writing smithi01698494-101 from 1651964 to 2053649 tid 2 2023-01-25T05:22:27.903 INFO:tasks.rados.rados.0.smithi016.stdout:101: writing smithi01698494-101 from 2713358 to 3483971 tid 3 2023-01-25T05:22:27.903 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.904 INFO:tasks.rados.rados.0.smithi016.stdout:86: finishing write tid 1 to smithi01698494-86 2023-01-25T05:22:27.904 INFO:tasks.rados.rados.0.smithi016.stdout:86: finishing write tid 2 to smithi01698494-86 2023-01-25T05:22:27.904 INFO:tasks.rados.rados.0.smithi016.stdout:86: finishing write tid 3 to smithi01698494-86 2023-01-25T05:22:27.904 INFO:tasks.rados.rados.0.smithi016.stdout:86: finishing write tid 4 to smithi01698494-86 2023-01-25T05:22:27.904 INFO:tasks.rados.rados.0.smithi016.stdout:86: finishing write tid 5 to smithi01698494-86 2023-01-25T05:22:27.905 INFO:tasks.rados.rados.0.smithi016.stdout:86: oid 86 updating version 0 to 43 2023-01-25T05:22:27.905 INFO:tasks.rados.rados.0.smithi016.stdout:86: oid 86 version 43 is already newer than 42 2023-01-25T05:22:27.905 INFO:tasks.rados.rados.0.smithi016.stdout:86: oid 86 version 43 is already newer than 41 2023-01-25T05:22:27.905 INFO:tasks.rados.rados.0.smithi016.stdout:86: oid 86 version 43 is already newer than 40 2023-01-25T05:22:27.905 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 86 v 43 (ObjNum 85 snap 0 seq_num 85) dirty exists 2023-01-25T05:22:27.906 INFO:tasks.rados.rados.0.smithi016.stdout:86: left oid 86 (ObjNum 85 snap 0 seq_num 85) 2023-01-25T05:22:27.906 INFO:tasks.rados.rados.0.smithi016.stdout:87: finishing write tid 1 to smithi01698494-87 2023-01-25T05:22:27.906 INFO:tasks.rados.rados.0.smithi016.stdout:87: finishing write tid 2 to smithi01698494-87 2023-01-25T05:22:27.906 INFO:tasks.rados.rados.0.smithi016.stdout:87: finishing write tid 3 to smithi01698494-87 2023-01-25T05:22:27.906 INFO:tasks.rados.rados.0.smithi016.stdout:87: finishing write tid 4 to smithi01698494-87 2023-01-25T05:22:27.907 INFO:tasks.rados.rados.0.smithi016.stdout:87: finishing write tid 5 to smithi01698494-87 2023-01-25T05:22:27.907 INFO:tasks.rados.rados.0.smithi016.stdout:87: oid 87 updating version 0 to 41 2023-01-25T05:22:27.907 INFO:tasks.rados.rados.0.smithi016.stdout:87: oid 87 updating version 41 to 43 2023-01-25T05:22:27.907 INFO:tasks.rados.rados.0.smithi016.stdout:87: oid 87 version 43 is already newer than 42 2023-01-25T05:22:27.907 INFO:tasks.rados.rados.0.smithi016.stdout:87: oid 87 updating version 43 to 44 2023-01-25T05:22:27.908 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 87 v 44 (ObjNum 86 snap 0 seq_num 86) dirty exists 2023-01-25T05:22:27.908 INFO:tasks.rados.rados.0.smithi016.stdout:87: left oid 87 (ObjNum 86 snap 0 seq_num 86) 2023-01-25T05:22:27.908 INFO:tasks.rados.rados.0.smithi016.stdout:88: finishing write tid 1 to smithi01698494-88 2023-01-25T05:22:27.909 INFO:tasks.rados.rados.0.smithi016.stdout:88: finishing write tid 2 to smithi01698494-88 2023-01-25T05:22:27.909 INFO:tasks.rados.rados.0.smithi016.stdout:88: finishing write tid 3 to smithi01698494-88 2023-01-25T05:22:27.909 INFO:tasks.rados.rados.0.smithi016.stdout:88: finishing write tid 4 to smithi01698494-88 2023-01-25T05:22:27.909 INFO:tasks.rados.rados.0.smithi016.stdout:88: finishing write tid 5 to smithi01698494-88 2023-01-25T05:22:27.909 INFO:tasks.rados.rados.0.smithi016.stdout:88: oid 88 updating version 0 to 46 2023-01-25T05:22:27.910 INFO:tasks.rados.rados.0.smithi016.stdout:88: oid 88 version 46 is already newer than 44 2023-01-25T05:22:27.910 INFO:tasks.rados.rados.0.smithi016.stdout:88: oid 88 version 46 is already newer than 45 2023-01-25T05:22:27.910 INFO:tasks.rados.rados.0.smithi016.stdout:88: oid 88 updating version 46 to 47 2023-01-25T05:22:27.910 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 88 v 47 (ObjNum 87 snap 0 seq_num 87) dirty exists 2023-01-25T05:22:27.910 INFO:tasks.rados.rados.0.smithi016.stdout:88: left oid 88 (ObjNum 87 snap 0 seq_num 87) 2023-01-25T05:22:27.911 INFO:tasks.rados.rados.0.smithi016.stdout:89: finishing write tid 1 to smithi01698494-89 2023-01-25T05:22:27.911 INFO:tasks.rados.rados.0.smithi016.stdout:89: finishing write tid 2 to smithi01698494-89 2023-01-25T05:22:27.911 INFO:tasks.rados.rados.0.smithi016.stdout:89: finishing write tid 3 to smithi01698494-89 2023-01-25T05:22:27.911 INFO:tasks.rados.rados.0.smithi016.stdout:89: finishing write tid 4 to smithi01698494-89 2023-01-25T05:22:27.911 INFO:tasks.rados.rados.0.smithi016.stdout:86: done (15 left) 2023-01-25T05:22:27.912 INFO:tasks.rados.rados.0.smithi016.stdout:87: done (14 left) 2023-01-25T05:22:27.912 INFO:tasks.rados.rados.0.smithi016.stdout:88: done (13 left) 2023-01-25T05:22:27.912 INFO:tasks.rados.rados.0.smithi016.stdout:102: write initial oid 102 2023-01-25T05:22:27.912 INFO:tasks.rados.rados.0.smithi016.stdout:102: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2023-01-25T05:22:27.913 INFO:tasks.rados.rados.0.smithi016.stdout:102: writing smithi01698494-102 from 497507 to 1210245 tid 1 2023-01-25T05:22:27.915 INFO:tasks.rados.rados.0.smithi016.stdout:102: writing smithi01698494-102 from 1710446 to 2372142 tid 2 2023-01-25T05:22:27.923 INFO:tasks.rados.rados.0.smithi016.stdout:102: writing smithi01698494-102 from 3140781 to 3697507 tid 3 2023-01-25T05:22:27.923 INFO:tasks.rados.rados.0.smithi016.stdout:103: write initial oid 103 2023-01-25T05:22:27.923 INFO:tasks.rados.rados.0.smithi016.stdout:103: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2023-01-25T05:22:27.930 INFO:tasks.rados.rados.0.smithi016.stdout:103: writing smithi01698494-103 from 514314 to 1302301 tid 1 2023-01-25T05:22:27.935 INFO:tasks.rados.rados.0.smithi016.stdout:103: writing smithi01698494-103 from 1768928 to 2174282 tid 2 2023-01-25T05:22:27.941 INFO:tasks.rados.rados.0.smithi016.stdout:103: writing smithi01698494-103 from 2768204 to 3277714 tid 3 2023-01-25T05:22:27.941 INFO:tasks.rados.rados.0.smithi016.stdout:104: write initial oid 104 2023-01-25T05:22:27.941 INFO:tasks.rados.rados.0.smithi016.stdout:104: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2023-01-25T05:22:27.946 INFO:tasks.rados.rados.0.smithi016.stdout:104: writing smithi01698494-104 from 531121 to 994357 tid 1 2023-01-25T05:22:27.954 INFO:tasks.rados.rados.0.smithi016.stdout:104: writing smithi01698494-104 from 1711057 to 2376422 tid 2 2023-01-25T05:22:27.961 INFO:tasks.rados.rados.0.smithi016.stdout:104: writing smithi01698494-104 from 3079274 to 3731121 tid 3 2023-01-25T05:22:27.961 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.962 INFO:tasks.rados.rados.0.smithi016.stdout:89: finishing write tid 5 to smithi01698494-89 2023-01-25T05:22:27.962 INFO:tasks.rados.rados.0.smithi016.stdout:89: oid 89 updating version 0 to 46 2023-01-25T05:22:27.962 INFO:tasks.rados.rados.0.smithi016.stdout:89: oid 89 updating version 46 to 48 2023-01-25T05:22:27.962 INFO:tasks.rados.rados.0.smithi016.stdout:89: oid 89 version 48 is already newer than 47 2023-01-25T05:22:27.962 INFO:tasks.rados.rados.0.smithi016.stdout:89: oid 89 version 48 is already newer than 45 2023-01-25T05:22:27.963 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 89 v 48 (ObjNum 88 snap 0 seq_num 88) dirty exists 2023-01-25T05:22:27.963 INFO:tasks.rados.rados.0.smithi016.stdout:89: left oid 89 (ObjNum 88 snap 0 seq_num 88) 2023-01-25T05:22:27.963 INFO:tasks.rados.rados.0.smithi016.stdout:89: done (15 left) 2023-01-25T05:22:27.963 INFO:tasks.rados.rados.0.smithi016.stdout:105: write initial oid 105 2023-01-25T05:22:27.963 INFO:tasks.rados.rados.0.smithi016.stdout:105: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2023-01-25T05:22:27.967 INFO:tasks.rados.rados.0.smithi016.stdout:105: writing smithi01698494-105 from 547928 to 1086413 tid 1 2023-01-25T05:22:27.973 INFO:tasks.rados.rados.0.smithi016.stdout:105: writing smithi01698494-105 from 1769539 to 2178562 tid 2 2023-01-25T05:22:27.979 INFO:tasks.rados.rados.0.smithi016.stdout:105: writing smithi01698494-105 from 2706697 to 3238751 tid 3 2023-01-25T05:22:27.980 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:27.980 INFO:tasks.rados.rados.0.smithi016.stdout:90: finishing write tid 1 to smithi01698494-90 2023-01-25T05:22:27.980 INFO:tasks.rados.rados.0.smithi016.stdout:90: finishing write tid 2 to smithi01698494-90 2023-01-25T05:22:27.980 INFO:tasks.rados.rados.0.smithi016.stdout:90: finishing write tid 3 to smithi01698494-90 2023-01-25T05:22:27.980 INFO:tasks.rados.rados.0.smithi016.stdout:90: finishing write tid 4 to smithi01698494-90 2023-01-25T05:22:27.981 INFO:tasks.rados.rados.0.smithi016.stdout:90: finishing write tid 5 to smithi01698494-90 2023-01-25T05:22:27.981 INFO:tasks.rados.rados.0.smithi016.stdout:90: oid 90 updating version 0 to 54 2023-01-25T05:22:27.981 INFO:tasks.rados.rados.0.smithi016.stdout:90: oid 90 version 54 is already newer than 53 2023-01-25T05:22:27.981 INFO:tasks.rados.rados.0.smithi016.stdout:90: oid 90 updating version 54 to 55 2023-01-25T05:22:27.981 INFO:tasks.rados.rados.0.smithi016.stdout:90: oid 90 version 55 is already newer than 52 2023-01-25T05:22:27.982 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 90 v 55 (ObjNum 89 snap 0 seq_num 89) dirty exists 2023-01-25T05:22:27.982 INFO:tasks.rados.rados.0.smithi016.stdout:90: left oid 90 (ObjNum 89 snap 0 seq_num 89) 2023-01-25T05:22:27.982 INFO:tasks.rados.rados.0.smithi016.stdout:91: finishing write tid 1 to smithi01698494-91 2023-01-25T05:22:27.982 INFO:tasks.rados.rados.0.smithi016.stdout:91: finishing write tid 2 to smithi01698494-91 2023-01-25T05:22:27.982 INFO:tasks.rados.rados.0.smithi016.stdout:91: finishing write tid 3 to smithi01698494-91 2023-01-25T05:22:27.983 INFO:tasks.rados.rados.0.smithi016.stdout:91: finishing write tid 4 to smithi01698494-91 2023-01-25T05:22:27.983 INFO:tasks.rados.rados.0.smithi016.stdout:91: finishing write tid 5 to smithi01698494-91 2023-01-25T05:22:27.983 INFO:tasks.rados.rados.0.smithi016.stdout:91: oid 91 updating version 0 to 57 2023-01-25T05:22:27.984 INFO:tasks.rados.rados.0.smithi016.stdout:91: oid 91 version 57 is already newer than 56 2023-01-25T05:22:27.984 INFO:tasks.rados.rados.0.smithi016.stdout:91: oid 91 updating version 57 to 58 2023-01-25T05:22:27.984 INFO:tasks.rados.rados.0.smithi016.stdout:91: oid 91 updating version 58 to 59 2023-01-25T05:22:27.984 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 91 v 59 (ObjNum 90 snap 0 seq_num 90) dirty exists 2023-01-25T05:22:27.985 INFO:tasks.rados.rados.0.smithi016.stdout:91: left oid 91 (ObjNum 90 snap 0 seq_num 90) 2023-01-25T05:22:27.985 INFO:tasks.rados.rados.0.smithi016.stdout:90: done (15 left) 2023-01-25T05:22:27.985 INFO:tasks.rados.rados.0.smithi016.stdout:91: done (14 left) 2023-01-25T05:22:27.985 INFO:tasks.rados.rados.0.smithi016.stdout:106: write initial oid 106 2023-01-25T05:22:27.985 INFO:tasks.rados.rados.0.smithi016.stdout:106: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2023-01-25T05:22:27.986 INFO:tasks.rados.rados.0.smithi016.stdout:106: writing smithi01698494-106 from 564735 to 1178469 tid 1 2023-01-25T05:22:27.992 INFO:tasks.rados.rados.0.smithi016.stdout:106: writing smithi01698494-106 from 1828021 to 2497055 tid 2 2023-01-25T05:22:27.999 INFO:tasks.rados.rados.0.smithi016.stdout:106: writing smithi01698494-106 from 3134120 to 3764735 tid 3 2023-01-25T05:22:28.000 INFO:tasks.rados.rados.0.smithi016.stdout:107: write initial oid 107 2023-01-25T05:22:28.000 INFO:tasks.rados.rados.0.smithi016.stdout:107: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2023-01-25T05:22:28.006 INFO:tasks.rados.rados.0.smithi016.stdout:107: writing smithi01698494-107 from 581542 to 1270525 tid 1 2023-01-25T05:22:28.012 INFO:tasks.rados.rados.0.smithi016.stdout:107: writing smithi01698494-107 from 1886503 to 2299195 tid 2 2023-01-25T05:22:28.019 INFO:tasks.rados.rados.0.smithi016.stdout:107: writing smithi01698494-107 from 2761543 to 3432494 tid 3 2023-01-25T05:22:28.019 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.020 INFO:tasks.rados.rados.0.smithi016.stdout:92: finishing write tid 1 to smithi01698494-92 2023-01-25T05:22:28.020 INFO:tasks.rados.rados.0.smithi016.stdout:92: finishing write tid 2 to smithi01698494-92 2023-01-25T05:22:28.020 INFO:tasks.rados.rados.0.smithi016.stdout:92: finishing write tid 3 to smithi01698494-92 2023-01-25T05:22:28.020 INFO:tasks.rados.rados.0.smithi016.stdout:92: finishing write tid 4 to smithi01698494-92 2023-01-25T05:22:28.021 INFO:tasks.rados.rados.0.smithi016.stdout:92: finishing write tid 5 to smithi01698494-92 2023-01-25T05:22:28.021 INFO:tasks.rados.rados.0.smithi016.stdout:92: oid 92 updating version 0 to 43 2023-01-25T05:22:28.021 INFO:tasks.rados.rados.0.smithi016.stdout:92: oid 92 updating version 43 to 44 2023-01-25T05:22:28.021 INFO:tasks.rados.rados.0.smithi016.stdout:92: oid 92 version 44 is already newer than 41 2023-01-25T05:22:28.021 INFO:tasks.rados.rados.0.smithi016.stdout:92: oid 92 version 44 is already newer than 42 2023-01-25T05:22:28.022 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 92 v 44 (ObjNum 91 snap 0 seq_num 91) dirty exists 2023-01-25T05:22:28.022 INFO:tasks.rados.rados.0.smithi016.stdout:92: left oid 92 (ObjNum 91 snap 0 seq_num 91) 2023-01-25T05:22:28.022 INFO:tasks.rados.rados.0.smithi016.stdout:93: finishing write tid 1 to smithi01698494-93 2023-01-25T05:22:28.022 INFO:tasks.rados.rados.0.smithi016.stdout:93: finishing write tid 2 to smithi01698494-93 2023-01-25T05:22:28.022 INFO:tasks.rados.rados.0.smithi016.stdout:93: finishing write tid 3 to smithi01698494-93 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: finishing write tid 4 to smithi01698494-93 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: finishing write tid 5 to smithi01698494-93 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: oid 93 updating version 0 to 51 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: oid 93 version 51 is already newer than 50 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: oid 93 version 51 is already newer than 48 2023-01-25T05:22:28.023 INFO:tasks.rados.rados.0.smithi016.stdout:93: oid 93 version 51 is already newer than 49 2023-01-25T05:22:28.024 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 93 v 51 (ObjNum 92 snap 0 seq_num 92) dirty exists 2023-01-25T05:22:28.024 INFO:tasks.rados.rados.0.smithi016.stdout:93: left oid 93 (ObjNum 92 snap 0 seq_num 92) 2023-01-25T05:22:28.024 INFO:tasks.rados.rados.0.smithi016.stdout:94: finishing write tid 1 to smithi01698494-94 2023-01-25T05:22:28.024 INFO:tasks.rados.rados.0.smithi016.stdout:94: finishing write tid 2 to smithi01698494-94 2023-01-25T05:22:28.024 INFO:tasks.rados.rados.0.smithi016.stdout:94: finishing write tid 3 to smithi01698494-94 2023-01-25T05:22:28.025 INFO:tasks.rados.rados.0.smithi016.stdout:94: finishing write tid 4 to smithi01698494-94 2023-01-25T05:22:28.025 INFO:tasks.rados.rados.0.smithi016.stdout:94: finishing write tid 5 to smithi01698494-94 2023-01-25T05:22:28.025 INFO:tasks.rados.rados.0.smithi016.stdout:94: oid 94 updating version 0 to 52 2023-01-25T05:22:28.025 INFO:tasks.rados.rados.0.smithi016.stdout:94: oid 94 version 52 is already newer than 51 2023-01-25T05:22:28.025 INFO:tasks.rados.rados.0.smithi016.stdout:94: oid 94 version 52 is already newer than 50 2023-01-25T05:22:28.026 INFO:tasks.rados.rados.0.smithi016.stdout:94: oid 94 version 52 is already newer than 49 2023-01-25T05:22:28.026 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 94 v 52 (ObjNum 93 snap 0 seq_num 93) dirty exists 2023-01-25T05:22:28.026 INFO:tasks.rados.rados.0.smithi016.stdout:94: left oid 94 (ObjNum 93 snap 0 seq_num 93) 2023-01-25T05:22:28.026 INFO:tasks.rados.rados.0.smithi016.stdout:95: finishing write tid 1 to smithi01698494-95 2023-01-25T05:22:28.026 INFO:tasks.rados.rados.0.smithi016.stdout:95: finishing write tid 2 to smithi01698494-95 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:95: finishing write tid 3 to smithi01698494-95 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:95: finishing write tid 4 to smithi01698494-95 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:92: done (15 left) 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:93: done (14 left) 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:94: done (13 left) 2023-01-25T05:22:28.027 INFO:tasks.rados.rados.0.smithi016.stdout:108: write initial oid 108 2023-01-25T05:22:28.028 INFO:tasks.rados.rados.0.smithi016.stdout:108: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2023-01-25T05:22:28.028 INFO:tasks.rados.rados.0.smithi016.stdout:108: writing smithi01698494-108 from 598349 to 1078934 tid 1 2023-01-25T05:22:28.031 INFO:tasks.rados.rados.0.smithi016.stdout:108: writing smithi01698494-108 from 1544985 to 2217688 tid 2 2023-01-25T05:22:28.037 INFO:tasks.rados.rados.0.smithi016.stdout:108: writing smithi01698494-108 from 2905319 to 3387542 tid 3 2023-01-25T05:22:28.037 INFO:tasks.rados.rados.0.smithi016.stdout:109: write initial oid 109 2023-01-25T05:22:28.038 INFO:tasks.rados.rados.0.smithi016.stdout:109: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2023-01-25T05:22:28.043 INFO:tasks.rados.rados.0.smithi016.stdout:109: writing smithi01698494-109 from 615156 to 1170990 tid 1 2023-01-25T05:22:28.048 INFO:tasks.rados.rados.0.smithi016.stdout:109: writing smithi01698494-109 from 1603467 to 2019828 tid 2 2023-01-25T05:22:28.057 INFO:tasks.rados.rados.0.smithi016.stdout:109: writing smithi01698494-109 from 2816389 to 3509884 tid 3 2023-01-25T05:22:28.057 INFO:tasks.rados.rados.0.smithi016.stdout:110: write initial oid 110 2023-01-25T05:22:28.057 INFO:tasks.rados.rados.0.smithi016.stdout:110: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2023-01-25T05:22:28.064 INFO:tasks.rados.rados.0.smithi016.stdout:110: writing smithi01698494-110 from 631963 to 1263046 tid 1 2023-01-25T05:22:28.071 INFO:tasks.rados.rados.0.smithi016.stdout:110: writing smithi01698494-110 from 2061949 to 2621968 tid 2 2023-01-25T05:22:28.078 INFO:tasks.rados.rados.0.smithi016.stdout:110: writing smithi01698494-110 from 3243812 to 3748579 tid 3 2023-01-25T05:22:28.078 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.078 INFO:tasks.rados.rados.0.smithi016.stdout:95: finishing write tid 5 to smithi01698494-95 2023-01-25T05:22:28.078 INFO:tasks.rados.rados.0.smithi016.stdout:95: oid 95 updating version 0 to 56 2023-01-25T05:22:28.079 INFO:tasks.rados.rados.0.smithi016.stdout:95: oid 95 version 56 is already newer than 55 2023-01-25T05:22:28.079 INFO:tasks.rados.rados.0.smithi016.stdout:95: oid 95 version 56 is already newer than 54 2023-01-25T05:22:28.079 INFO:tasks.rados.rados.0.smithi016.stdout:95: oid 95 version 56 is already newer than 53 2023-01-25T05:22:28.079 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 95 v 56 (ObjNum 94 snap 0 seq_num 94) dirty exists 2023-01-25T05:22:28.080 INFO:tasks.rados.rados.0.smithi016.stdout:95: left oid 95 (ObjNum 94 snap 0 seq_num 94) 2023-01-25T05:22:28.080 INFO:tasks.rados.rados.0.smithi016.stdout:95: done (15 left) 2023-01-25T05:22:28.080 INFO:tasks.rados.rados.0.smithi016.stdout:111: write initial oid 111 2023-01-25T05:22:28.080 INFO:tasks.rados.rados.0.smithi016.stdout:111: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2023-01-25T05:22:28.085 INFO:tasks.rados.rados.0.smithi016.stdout:111: writing smithi01698494-111 from 648770 to 1355102 tid 1 2023-01-25T05:22:28.091 INFO:tasks.rados.rados.0.smithi016.stdout:111: writing smithi01698494-111 from 2120431 to 2540461 tid 2 2023-01-25T05:22:28.098 INFO:tasks.rados.rados.0.smithi016.stdout:111: writing smithi01698494-111 from 3271235 to 3703627 tid 3 2023-01-25T05:22:28.098 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.099 INFO:tasks.rados.rados.0.smithi016.stdout:96: finishing write tid 1 to smithi01698494-96 2023-01-25T05:22:28.099 INFO:tasks.rados.rados.0.smithi016.stdout:96: finishing write tid 2 to smithi01698494-96 2023-01-25T05:22:28.099 INFO:tasks.rados.rados.0.smithi016.stdout:96: finishing write tid 3 to smithi01698494-96 2023-01-25T05:22:28.099 INFO:tasks.rados.rados.0.smithi016.stdout:96: finishing write tid 4 to smithi01698494-96 2023-01-25T05:22:28.100 INFO:tasks.rados.rados.0.smithi016.stdout:96: finishing write tid 5 to smithi01698494-96 2023-01-25T05:22:28.100 INFO:tasks.rados.rados.0.smithi016.stdout:96: oid 96 updating version 0 to 62 2023-01-25T05:22:28.100 INFO:tasks.rados.rados.0.smithi016.stdout:96: oid 96 version 62 is already newer than 60 2023-01-25T05:22:28.100 INFO:tasks.rados.rados.0.smithi016.stdout:96: oid 96 version 62 is already newer than 61 2023-01-25T05:22:28.100 INFO:tasks.rados.rados.0.smithi016.stdout:96: oid 96 updating version 62 to 63 2023-01-25T05:22:28.101 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 96 v 63 (ObjNum 95 snap 0 seq_num 95) dirty exists 2023-01-25T05:22:28.101 INFO:tasks.rados.rados.0.smithi016.stdout:96: left oid 96 (ObjNum 95 snap 0 seq_num 95) 2023-01-25T05:22:28.101 INFO:tasks.rados.rados.0.smithi016.stdout:97: finishing write tid 1 to smithi01698494-97 2023-01-25T05:22:28.101 INFO:tasks.rados.rados.0.smithi016.stdout:96: done (15 left) 2023-01-25T05:22:28.101 INFO:tasks.rados.rados.0.smithi016.stdout:112: write initial oid 112 2023-01-25T05:22:28.102 INFO:tasks.rados.rados.0.smithi016.stdout:112: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2023-01-25T05:22:28.105 INFO:tasks.rados.rados.0.smithi016.stdout:112: writing smithi01698494-112 from 665577 to 1447158 tid 1 2023-01-25T05:22:28.112 INFO:tasks.rados.rados.0.smithi016.stdout:112: writing smithi01698494-112 from 2062560 to 2626248 tid 2 2023-01-25T05:22:28.119 INFO:tasks.rados.rados.0.smithi016.stdout:112: writing smithi01698494-112 from 3182305 to 3825969 tid 3 2023-01-25T05:22:28.120 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.120 INFO:tasks.rados.rados.0.smithi016.stdout:97: finishing write tid 2 to smithi01698494-97 2023-01-25T05:22:28.120 INFO:tasks.rados.rados.0.smithi016.stdout:97: finishing write tid 3 to smithi01698494-97 2023-01-25T05:22:28.120 INFO:tasks.rados.rados.0.smithi016.stdout:97: finishing write tid 4 to smithi01698494-97 2023-01-25T05:22:28.120 INFO:tasks.rados.rados.0.smithi016.stdout:97: finishing write tid 5 to smithi01698494-97 2023-01-25T05:22:28.121 INFO:tasks.rados.rados.0.smithi016.stdout:97: oid 97 updating version 0 to 64 2023-01-25T05:22:28.121 INFO:tasks.rados.rados.0.smithi016.stdout:97: oid 97 updating version 64 to 67 2023-01-25T05:22:28.121 INFO:tasks.rados.rados.0.smithi016.stdout:97: oid 97 version 67 is already newer than 66 2023-01-25T05:22:28.121 INFO:tasks.rados.rados.0.smithi016.stdout:97: oid 97 version 67 is already newer than 65 2023-01-25T05:22:28.121 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 97 v 67 (ObjNum 96 snap 0 seq_num 96) dirty exists 2023-01-25T05:22:28.122 INFO:tasks.rados.rados.0.smithi016.stdout:97: left oid 97 (ObjNum 96 snap 0 seq_num 96) 2023-01-25T05:22:28.122 INFO:tasks.rados.rados.0.smithi016.stdout:98: finishing write tid 1 to smithi01698494-98 2023-01-25T05:22:28.122 INFO:tasks.rados.rados.0.smithi016.stdout:97: done (15 left) 2023-01-25T05:22:28.122 INFO:tasks.rados.rados.0.smithi016.stdout:113: write initial oid 113 2023-01-25T05:22:28.122 INFO:tasks.rados.rados.0.smithi016.stdout:113: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2023-01-25T05:22:28.125 INFO:tasks.rados.rados.0.smithi016.stdout:113: writing smithi01698494-113 from 682384 to 1139214 tid 1 2023-01-25T05:22:28.130 INFO:tasks.rados.rados.0.smithi016.stdout:113: writing smithi01698494-113 from 1721042 to 2144741 tid 2 2023-01-25T05:22:28.136 INFO:tasks.rados.rados.0.smithi016.stdout:113: writing smithi01698494-113 from 2809728 to 3264664 tid 3 2023-01-25T05:22:28.140 INFO:tasks.rados.rados.0.smithi016.stdout:113: writing smithi01698494-113 from 3731357 to 3882384 tid 4 2023-01-25T05:22:28.140 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.141 INFO:tasks.rados.rados.0.smithi016.stdout:98: finishing write tid 2 to smithi01698494-98 2023-01-25T05:22:28.141 INFO:tasks.rados.rados.0.smithi016.stdout:98: finishing write tid 3 to smithi01698494-98 2023-01-25T05:22:28.141 INFO:tasks.rados.rados.0.smithi016.stdout:98: finishing write tid 4 to smithi01698494-98 2023-01-25T05:22:28.141 INFO:tasks.rados.rados.0.smithi016.stdout:98: finishing write tid 5 to smithi01698494-98 2023-01-25T05:22:28.141 INFO:tasks.rados.rados.0.smithi016.stdout:98: oid 98 updating version 0 to 69 2023-01-25T05:22:28.142 INFO:tasks.rados.rados.0.smithi016.stdout:98: oid 98 version 69 is already newer than 68 2023-01-25T05:22:28.142 INFO:tasks.rados.rados.0.smithi016.stdout:98: oid 98 updating version 69 to 71 2023-01-25T05:22:28.142 INFO:tasks.rados.rados.0.smithi016.stdout:98: oid 98 version 71 is already newer than 70 2023-01-25T05:22:28.142 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 98 v 71 (ObjNum 97 snap 0 seq_num 97) dirty exists 2023-01-25T05:22:28.142 INFO:tasks.rados.rados.0.smithi016.stdout:98: left oid 98 (ObjNum 97 snap 0 seq_num 97) 2023-01-25T05:22:28.143 INFO:tasks.rados.rados.0.smithi016.stdout:99: finishing write tid 1 to smithi01698494-99 2023-01-25T05:22:28.143 INFO:tasks.rados.rados.0.smithi016.stdout:98: done (15 left) 2023-01-25T05:22:28.143 INFO:tasks.rados.rados.0.smithi016.stdout:114: write initial oid 114 2023-01-25T05:22:28.143 INFO:tasks.rados.rados.0.smithi016.stdout:114: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2023-01-25T05:22:28.147 INFO:tasks.rados.rados.0.smithi016.stdout:114: writing smithi01698494-114 from 699191 to 1231270 tid 1 2023-01-25T05:22:28.153 INFO:tasks.rados.rados.0.smithi016.stdout:114: writing smithi01698494-114 from 1779524 to 2346881 tid 2 2023-01-25T05:22:28.160 INFO:tasks.rados.rados.0.smithi016.stdout:114: writing smithi01698494-114 from 2837151 to 3503359 tid 3 2023-01-25T05:22:28.160 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.160 INFO:tasks.rados.rados.0.smithi016.stdout:99: finishing write tid 2 to smithi01698494-99 2023-01-25T05:22:28.161 INFO:tasks.rados.rados.0.smithi016.stdout:99: finishing write tid 3 to smithi01698494-99 2023-01-25T05:22:28.161 INFO:tasks.rados.rados.0.smithi016.stdout:99: finishing write tid 4 to smithi01698494-99 2023-01-25T05:22:28.161 INFO:tasks.rados.rados.0.smithi016.stdout:99: finishing write tid 5 to smithi01698494-99 2023-01-25T05:22:28.161 INFO:tasks.rados.rados.0.smithi016.stdout:99: oid 99 updating version 0 to 72 2023-01-25T05:22:28.161 INFO:tasks.rados.rados.0.smithi016.stdout:99: oid 99 updating version 72 to 74 2023-01-25T05:22:28.162 INFO:tasks.rados.rados.0.smithi016.stdout:99: oid 99 version 74 is already newer than 73 2023-01-25T05:22:28.162 INFO:tasks.rados.rados.0.smithi016.stdout:99: oid 99 updating version 74 to 75 2023-01-25T05:22:28.162 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 99 v 75 (ObjNum 98 snap 0 seq_num 98) dirty exists 2023-01-25T05:22:28.162 INFO:tasks.rados.rados.0.smithi016.stdout:99: left oid 99 (ObjNum 98 snap 0 seq_num 98) 2023-01-25T05:22:28.162 INFO:tasks.rados.rados.0.smithi016.stdout:100: finishing write tid 1 to smithi01698494-100 2023-01-25T05:22:28.163 INFO:tasks.rados.rados.0.smithi016.stdout:99: done (15 left) 2023-01-25T05:22:28.163 INFO:tasks.rados.rados.0.smithi016.stdout:115: write initial oid 115 2023-01-25T05:22:28.163 INFO:tasks.rados.rados.0.smithi016.stdout:115: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2023-01-25T05:22:28.167 INFO:tasks.rados.rados.0.smithi016.stdout:115: writing smithi01698494-115 from 715998 to 1323326 tid 1 2023-01-25T05:22:28.172 INFO:tasks.rados.rados.0.smithi016.stdout:115: writing smithi01698494-115 from 1838006 to 2265374 tid 2 2023-01-25T05:22:28.178 INFO:tasks.rados.rados.0.smithi016.stdout:115: writing smithi01698494-115 from 2864574 to 3342054 tid 3 2023-01-25T05:22:28.182 INFO:tasks.rados.rados.0.smithi016.stdout:115: writing smithi01698494-115 from 3863835 to 3915998 tid 4 2023-01-25T05:22:28.182 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.183 INFO:tasks.rados.rados.0.smithi016.stdout:100: finishing write tid 2 to smithi01698494-100 2023-01-25T05:22:28.183 INFO:tasks.rados.rados.0.smithi016.stdout:100: finishing write tid 3 to smithi01698494-100 2023-01-25T05:22:28.183 INFO:tasks.rados.rados.0.smithi016.stdout:100: finishing write tid 4 to smithi01698494-100 2023-01-25T05:22:28.183 INFO:tasks.rados.rados.0.smithi016.stdout:100: finishing write tid 5 to smithi01698494-100 2023-01-25T05:22:28.184 INFO:tasks.rados.rados.0.smithi016.stdout:100: oid 100 updating version 0 to 78 2023-01-25T05:22:28.184 INFO:tasks.rados.rados.0.smithi016.stdout:100: oid 100 updating version 78 to 79 2023-01-25T05:22:28.184 INFO:tasks.rados.rados.0.smithi016.stdout:100: oid 100 version 79 is already newer than 76 2023-01-25T05:22:28.184 INFO:tasks.rados.rados.0.smithi016.stdout:100: oid 100 version 79 is already newer than 77 2023-01-25T05:22:28.184 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 100 v 79 (ObjNum 99 snap 0 seq_num 99) dirty exists 2023-01-25T05:22:28.185 INFO:tasks.rados.rados.0.smithi016.stdout:100: left oid 100 (ObjNum 99 snap 0 seq_num 99) 2023-01-25T05:22:28.185 INFO:tasks.rados.rados.0.smithi016.stdout:101: finishing write tid 1 to smithi01698494-101 2023-01-25T05:22:28.185 INFO:tasks.rados.rados.0.smithi016.stdout:100: done (15 left) 2023-01-25T05:22:28.185 INFO:tasks.rados.rados.0.smithi016.stdout:116: write initial oid 116 2023-01-25T05:22:28.185 INFO:tasks.rados.rados.0.smithi016.stdout:116: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2023-01-25T05:22:28.191 INFO:tasks.rados.rados.0.smithi016.stdout:116: writing smithi01698494-116 from 732805 to 1531735 tid 1 2023-01-25T05:22:28.198 INFO:tasks.rados.rados.0.smithi016.stdout:116: writing smithi01698494-116 from 2296488 to 2867514 tid 2 2023-01-25T05:22:28.205 INFO:tasks.rados.rados.0.smithi016.stdout:116: writing smithi01698494-116 from 3291997 to 3697102 tid 3 2023-01-25T05:22:28.205 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.205 INFO:tasks.rados.rados.0.smithi016.stdout:101: finishing write tid 2 to smithi01698494-101 2023-01-25T05:22:28.206 INFO:tasks.rados.rados.0.smithi016.stdout:101: finishing write tid 3 to smithi01698494-101 2023-01-25T05:22:28.206 INFO:tasks.rados.rados.0.smithi016.stdout:101: finishing write tid 4 to smithi01698494-101 2023-01-25T05:22:28.206 INFO:tasks.rados.rados.0.smithi016.stdout:101: finishing write tid 5 to smithi01698494-101 2023-01-25T05:22:28.206 INFO:tasks.rados.rados.0.smithi016.stdout:101: oid 101 updating version 0 to 59 2023-01-25T05:22:28.207 INFO:tasks.rados.rados.0.smithi016.stdout:101: oid 101 version 59 is already newer than 58 2023-01-25T05:22:28.207 INFO:tasks.rados.rados.0.smithi016.stdout:101: oid 101 updating version 59 to 60 2023-01-25T05:22:28.207 INFO:tasks.rados.rados.0.smithi016.stdout:101: oid 101 version 60 is already newer than 57 2023-01-25T05:22:28.207 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 101 v 60 (ObjNum 100 snap 0 seq_num 100) dirty exists 2023-01-25T05:22:28.208 INFO:tasks.rados.rados.0.smithi016.stdout:101: left oid 101 (ObjNum 100 snap 0 seq_num 100) 2023-01-25T05:22:28.208 INFO:tasks.rados.rados.0.smithi016.stdout:102: finishing write tid 1 to smithi01698494-102 2023-01-25T05:22:28.208 INFO:tasks.rados.rados.0.smithi016.stdout:102: finishing write tid 2 to smithi01698494-102 2023-01-25T05:22:28.208 INFO:tasks.rados.rados.0.smithi016.stdout:102: finishing write tid 3 to smithi01698494-102 2023-01-25T05:22:28.208 INFO:tasks.rados.rados.0.smithi016.stdout:102: finishing write tid 4 to smithi01698494-102 2023-01-25T05:22:28.209 INFO:tasks.rados.rados.0.smithi016.stdout:102: finishing write tid 5 to smithi01698494-102 2023-01-25T05:22:28.209 INFO:tasks.rados.rados.0.smithi016.stdout:102: oid 102 updating version 0 to 54 2023-01-25T05:22:28.209 INFO:tasks.rados.rados.0.smithi016.stdout:102: oid 102 version 54 is already newer than 53 2023-01-25T05:22:28.209 INFO:tasks.rados.rados.0.smithi016.stdout:102: oid 102 version 54 is already newer than 52 2023-01-25T05:22:28.209 INFO:tasks.rados.rados.0.smithi016.stdout:102: oid 102 updating version 54 to 55 2023-01-25T05:22:28.210 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 102 v 55 (ObjNum 101 snap 0 seq_num 101) dirty exists 2023-01-25T05:22:28.210 INFO:tasks.rados.rados.0.smithi016.stdout:102: left oid 102 (ObjNum 101 snap 0 seq_num 101) 2023-01-25T05:22:28.210 INFO:tasks.rados.rados.0.smithi016.stdout:103: finishing write tid 1 to smithi01698494-103 2023-01-25T05:22:28.210 INFO:tasks.rados.rados.0.smithi016.stdout:103: finishing write tid 2 to smithi01698494-103 2023-01-25T05:22:28.210 INFO:tasks.rados.rados.0.smithi016.stdout:103: finishing write tid 3 to smithi01698494-103 2023-01-25T05:22:28.211 INFO:tasks.rados.rados.0.smithi016.stdout:103: finishing write tid 4 to smithi01698494-103 2023-01-25T05:22:28.211 INFO:tasks.rados.rados.0.smithi016.stdout:103: finishing write tid 5 to smithi01698494-103 2023-01-25T05:22:28.211 INFO:tasks.rados.rados.0.smithi016.stdout:103: oid 103 updating version 0 to 63 2023-01-25T05:22:28.211 INFO:tasks.rados.rados.0.smithi016.stdout:103: oid 103 version 63 is already newer than 61 2023-01-25T05:22:28.212 INFO:tasks.rados.rados.0.smithi016.stdout:103: oid 103 version 63 is already newer than 62 2023-01-25T05:22:28.212 INFO:tasks.rados.rados.0.smithi016.stdout:103: oid 103 updating version 63 to 64 2023-01-25T05:22:28.212 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 103 v 64 (ObjNum 102 snap 0 seq_num 102) dirty exists 2023-01-25T05:22:28.212 INFO:tasks.rados.rados.0.smithi016.stdout:103: left oid 103 (ObjNum 102 snap 0 seq_num 102) 2023-01-25T05:22:28.213 INFO:tasks.rados.rados.0.smithi016.stdout:104: finishing write tid 1 to smithi01698494-104 2023-01-25T05:22:28.213 INFO:tasks.rados.rados.0.smithi016.stdout:104: finishing write tid 2 to smithi01698494-104 2023-01-25T05:22:28.213 INFO:tasks.rados.rados.0.smithi016.stdout:104: finishing write tid 3 to smithi01698494-104 2023-01-25T05:22:28.213 INFO:tasks.rados.rados.0.smithi016.stdout:104: finishing write tid 4 to smithi01698494-104 2023-01-25T05:22:28.213 INFO:tasks.rados.rados.0.smithi016.stdout:104: finishing write tid 5 to smithi01698494-104 2023-01-25T05:22:28.214 INFO:tasks.rados.rados.0.smithi016.stdout:104: oid 104 updating version 0 to 45 2023-01-25T05:22:28.214 INFO:tasks.rados.rados.0.smithi016.stdout:104: oid 104 updating version 45 to 46 2023-01-25T05:22:28.214 INFO:tasks.rados.rados.0.smithi016.stdout:104: oid 104 updating version 46 to 48 2023-01-25T05:22:28.214 INFO:tasks.rados.rados.0.smithi016.stdout:104: oid 104 version 48 is already newer than 47 2023-01-25T05:22:28.214 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 104 v 48 (ObjNum 103 snap 0 seq_num 103) dirty exists 2023-01-25T05:22:28.215 INFO:tasks.rados.rados.0.smithi016.stdout:104: left oid 104 (ObjNum 103 snap 0 seq_num 103) 2023-01-25T05:22:28.215 INFO:tasks.rados.rados.0.smithi016.stdout:105: finishing write tid 1 to smithi01698494-105 2023-01-25T05:22:28.215 INFO:tasks.rados.rados.0.smithi016.stdout:105: finishing write tid 2 to smithi01698494-105 2023-01-25T05:22:28.215 INFO:tasks.rados.rados.0.smithi016.stdout:105: finishing write tid 3 to smithi01698494-105 2023-01-25T05:22:28.215 INFO:tasks.rados.rados.0.smithi016.stdout:105: finishing write tid 4 to smithi01698494-105 2023-01-25T05:22:28.216 INFO:tasks.rados.rados.0.smithi016.stdout:105: finishing write tid 5 to smithi01698494-105 2023-01-25T05:22:28.216 INFO:tasks.rados.rados.0.smithi016.stdout:105: oid 105 updating version 0 to 82 2023-01-25T05:22:28.216 INFO:tasks.rados.rados.0.smithi016.stdout:105: oid 105 version 82 is already newer than 80 2023-01-25T05:22:28.216 INFO:tasks.rados.rados.0.smithi016.stdout:105: oid 105 updating version 82 to 83 2023-01-25T05:22:28.216 INFO:tasks.rados.rados.0.smithi016.stdout:105: oid 105 version 83 is already newer than 81 2023-01-25T05:22:28.217 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 105 v 83 (ObjNum 104 snap 0 seq_num 104) dirty exists 2023-01-25T05:22:28.217 INFO:tasks.rados.rados.0.smithi016.stdout:105: left oid 105 (ObjNum 104 snap 0 seq_num 104) 2023-01-25T05:22:28.217 INFO:tasks.rados.rados.0.smithi016.stdout:106: finishing write tid 1 to smithi01698494-106 2023-01-25T05:22:28.217 INFO:tasks.rados.rados.0.smithi016.stdout:106: finishing write tid 2 to smithi01698494-106 2023-01-25T05:22:28.217 INFO:tasks.rados.rados.0.smithi016.stdout:106: finishing write tid 3 to smithi01698494-106 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: finishing write tid 4 to smithi01698494-106 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: finishing write tid 5 to smithi01698494-106 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: oid 106 updating version 0 to 52 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: oid 106 version 52 is already newer than 49 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: oid 106 version 52 is already newer than 51 2023-01-25T05:22:28.218 INFO:tasks.rados.rados.0.smithi016.stdout:106: oid 106 version 52 is already newer than 50 2023-01-25T05:22:28.219 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 106 v 52 (ObjNum 105 snap 0 seq_num 105) dirty exists 2023-01-25T05:22:28.219 INFO:tasks.rados.rados.0.smithi016.stdout:106: left oid 106 (ObjNum 105 snap 0 seq_num 105) 2023-01-25T05:22:28.219 INFO:tasks.rados.rados.0.smithi016.stdout:107: finishing write tid 1 to smithi01698494-107 2023-01-25T05:22:28.219 INFO:tasks.rados.rados.0.smithi016.stdout:107: finishing write tid 2 to smithi01698494-107 2023-01-25T05:22:28.219 INFO:tasks.rados.rados.0.smithi016.stdout:101: done (15 left) 2023-01-25T05:22:28.220 INFO:tasks.rados.rados.0.smithi016.stdout:102: done (14 left) 2023-01-25T05:22:28.220 INFO:tasks.rados.rados.0.smithi016.stdout:103: done (13 left) 2023-01-25T05:22:28.220 INFO:tasks.rados.rados.0.smithi016.stdout:104: done (12 left) 2023-01-25T05:22:28.220 INFO:tasks.rados.rados.0.smithi016.stdout:105: done (11 left) 2023-01-25T05:22:28.221 INFO:tasks.rados.rados.0.smithi016.stdout:106: done (10 left) 2023-01-25T05:22:28.221 INFO:tasks.rados.rados.0.smithi016.stdout:117: write initial oid 117 2023-01-25T05:22:28.221 INFO:tasks.rados.rados.0.smithi016.stdout:117: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2023-01-25T05:22:28.222 INFO:tasks.rados.rados.0.smithi016.stdout:117: writing smithi01698494-117 from 749612 to 1223791 tid 1 2023-01-25T05:22:28.222 INFO:tasks.rados.rados.0.smithi016.stdout:117: writing smithi01698494-117 from 1954970 to 2386007 tid 2 2023-01-25T05:22:28.223 INFO:tasks.rados.rados.0.smithi016.stdout:117: writing smithi01698494-117 from 2919420 to 3535797 tid 3 2023-01-25T05:22:28.223 INFO:tasks.rados.rados.0.smithi016.stdout:118: write initial oid 118 2023-01-25T05:22:28.224 INFO:tasks.rados.rados.0.smithi016.stdout:118: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2023-01-25T05:22:28.230 INFO:tasks.rados.rados.0.smithi016.stdout:118: writing smithi01698494-118 from 766419 to 1315847 tid 1 2023-01-25T05:22:28.238 INFO:tasks.rados.rados.0.smithi016.stdout:118: writing smithi01698494-118 from 2013452 to 2588147 tid 2 2023-01-25T05:22:28.244 INFO:tasks.rados.rados.0.smithi016.stdout:118: writing smithi01698494-118 from 3346843 to 3774492 tid 3 2023-01-25T05:22:28.245 INFO:tasks.rados.rados.0.smithi016.stdout:119: write initial oid 119 2023-01-25T05:22:28.245 INFO:tasks.rados.rados.0.smithi016.stdout:119: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2023-01-25T05:22:28.252 INFO:tasks.rados.rados.0.smithi016.stdout:119: writing smithi01698494-119 from 783226 to 1407903 tid 1 2023-01-25T05:22:28.258 INFO:tasks.rados.rados.0.smithi016.stdout:119: writing smithi01698494-119 from 2071934 to 2506640 tid 2 2023-01-25T05:22:28.265 INFO:tasks.rados.rados.0.smithi016.stdout:119: writing smithi01698494-119 from 2974266 to 3613187 tid 3 2023-01-25T05:22:28.265 INFO:tasks.rados.rados.0.smithi016.stdout:120: write initial oid 120 2023-01-25T05:22:28.265 INFO:tasks.rados.rados.0.smithi016.stdout:120: seq_num 119 ranges {400033=699926,1614063=385970} 2023-01-25T05:22:28.271 INFO:tasks.rados.rados.0.smithi016.stdout:120: writing smithi01698494-120 from 400033 to 1099959 tid 1 2023-01-25T05:22:28.276 INFO:tasks.rados.rados.0.smithi016.stdout:120: writing smithi01698494-120 from 1614063 to 2000033 tid 2 2023-01-25T05:22:28.276 INFO:tasks.rados.rados.0.smithi016.stdout:121: write initial oid 121 2023-01-25T05:22:28.277 INFO:tasks.rados.rados.0.smithi016.stdout:121: seq_num 120 ranges {416840=775175,1672545=344295} 2023-01-25T05:22:28.283 INFO:tasks.rados.rados.0.smithi016.stdout:121: writing smithi01698494-121 from 416840 to 1192015 tid 1 2023-01-25T05:22:28.288 INFO:tasks.rados.rados.0.smithi016.stdout:121: writing smithi01698494-121 from 1672545 to 2016840 tid 2 2023-01-25T05:22:28.288 INFO:tasks.rados.rados.0.smithi016.stdout:122: write initial oid 122 2023-01-25T05:22:28.288 INFO:tasks.rados.rados.0.smithi016.stdout:122: seq_num 121 ranges {433647=450424,1331027=582033} 2023-01-25T05:22:28.293 INFO:tasks.rados.rados.0.smithi016.stdout:122: writing smithi01698494-122 from 433647 to 884071 tid 1 2023-01-25T05:22:28.299 INFO:tasks.rados.rados.0.smithi016.stdout:122: writing smithi01698494-122 from 1331027 to 1913060 tid 2 2023-01-25T05:22:28.300 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.300 INFO:tasks.rados.rados.0.smithi016.stdout:107: finishing write tid 3 to smithi01698494-107 2023-01-25T05:22:28.300 INFO:tasks.rados.rados.0.smithi016.stdout:107: finishing write tid 4 to smithi01698494-107 2023-01-25T05:22:28.300 INFO:tasks.rados.rados.0.smithi016.stdout:107: finishing write tid 5 to smithi01698494-107 2023-01-25T05:22:28.300 INFO:tasks.rados.rados.0.smithi016.stdout:107: oid 107 updating version 0 to 56 2023-01-25T05:22:28.301 INFO:tasks.rados.rados.0.smithi016.stdout:107: oid 107 updating version 56 to 58 2023-01-25T05:22:28.301 INFO:tasks.rados.rados.0.smithi016.stdout:107: oid 107 version 58 is already newer than 57 2023-01-25T05:22:28.301 INFO:tasks.rados.rados.0.smithi016.stdout:107: oid 107 updating version 58 to 59 2023-01-25T05:22:28.301 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 107 v 59 (ObjNum 106 snap 0 seq_num 106) dirty exists 2023-01-25T05:22:28.301 INFO:tasks.rados.rados.0.smithi016.stdout:107: left oid 107 (ObjNum 106 snap 0 seq_num 106) 2023-01-25T05:22:28.302 INFO:tasks.rados.rados.0.smithi016.stdout:108: finishing write tid 1 to smithi01698494-108 2023-01-25T05:22:28.302 INFO:tasks.rados.rados.0.smithi016.stdout:108: finishing write tid 2 to smithi01698494-108 2023-01-25T05:22:28.302 INFO:tasks.rados.rados.0.smithi016.stdout:108: finishing write tid 3 to smithi01698494-108 2023-01-25T05:22:28.302 INFO:tasks.rados.rados.0.smithi016.stdout:108: finishing write tid 4 to smithi01698494-108 2023-01-25T05:22:28.302 INFO:tasks.rados.rados.0.smithi016.stdout:107: done (15 left) 2023-01-25T05:22:28.303 INFO:tasks.rados.rados.0.smithi016.stdout:123: write initial oid 123 2023-01-25T05:22:28.303 INFO:tasks.rados.rados.0.smithi016.stdout:123: seq_num 122 ranges {450454=642026,1505862=544592} 2023-01-25T05:22:28.305 INFO:tasks.rados.rados.0.smithi016.stdout:123: writing smithi01698494-123 from 450454 to 1092480 tid 1 2023-01-25T05:22:28.311 INFO:tasks.rados.rados.0.smithi016.stdout:123: writing smithi01698494-123 from 1505862 to 2050454 tid 2 2023-01-25T05:22:28.311 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.312 INFO:tasks.rados.rados.0.smithi016.stdout:108: finishing write tid 5 to smithi01698494-108 2023-01-25T05:22:28.312 INFO:tasks.rados.rados.0.smithi016.stdout:108: oid 108 updating version 0 to 84 2023-01-25T05:22:28.312 INFO:tasks.rados.rados.0.smithi016.stdout:108: oid 108 updating version 84 to 85 2023-01-25T05:22:28.312 INFO:tasks.rados.rados.0.smithi016.stdout:108: oid 108 updating version 85 to 87 2023-01-25T05:22:28.312 INFO:tasks.rados.rados.0.smithi016.stdout:108: oid 108 version 87 is already newer than 86 2023-01-25T05:22:28.313 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 108 v 87 (ObjNum 107 snap 0 seq_num 107) dirty exists 2023-01-25T05:22:28.313 INFO:tasks.rados.rados.0.smithi016.stdout:108: left oid 108 (ObjNum 107 snap 0 seq_num 107) 2023-01-25T05:22:28.313 INFO:tasks.rados.rados.0.smithi016.stdout:108: done (15 left) 2023-01-25T05:22:28.313 INFO:tasks.rados.rados.0.smithi016.stdout:124: write initial oid 124 2023-01-25T05:22:28.313 INFO:tasks.rados.rados.0.smithi016.stdout:124: seq_num 123 ranges {467261=717275,1847991=219270} 2023-01-25T05:22:28.317 INFO:tasks.rados.rados.0.smithi016.stdout:124: writing smithi01698494-124 from 467261 to 1184536 tid 1 2023-01-25T05:22:28.323 INFO:tasks.rados.rados.0.smithi016.stdout:124: writing smithi01698494-124 from 1847991 to 2067261 tid 2 2023-01-25T05:22:28.323 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.323 INFO:tasks.rados.rados.0.smithi016.stdout:109: finishing write tid 1 to smithi01698494-109 2023-01-25T05:22:28.323 INFO:tasks.rados.rados.0.smithi016.stdout:109: finishing write tid 2 to smithi01698494-109 2023-01-25T05:22:28.324 INFO:tasks.rados.rados.0.smithi016.stdout:109: finishing write tid 3 to smithi01698494-109 2023-01-25T05:22:28.324 INFO:tasks.rados.rados.0.smithi016.stdout:109: finishing write tid 4 to smithi01698494-109 2023-01-25T05:22:28.324 INFO:tasks.rados.rados.0.smithi016.stdout:109: finishing write tid 5 to smithi01698494-109 2023-01-25T05:22:28.324 INFO:tasks.rados.rados.0.smithi016.stdout:109: oid 109 updating version 0 to 90 2023-01-25T05:22:28.324 INFO:tasks.rados.rados.0.smithi016.stdout:109: oid 109 version 90 is already newer than 89 2023-01-25T05:22:28.325 INFO:tasks.rados.rados.0.smithi016.stdout:109: oid 109 updating version 90 to 91 2023-01-25T05:22:28.325 INFO:tasks.rados.rados.0.smithi016.stdout:109: oid 109 version 91 is already newer than 88 2023-01-25T05:22:28.325 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 109 v 91 (ObjNum 108 snap 0 seq_num 108) dirty exists 2023-01-25T05:22:28.325 INFO:tasks.rados.rados.0.smithi016.stdout:109: left oid 109 (ObjNum 108 snap 0 seq_num 108) 2023-01-25T05:22:28.326 INFO:tasks.rados.rados.0.smithi016.stdout:110: finishing write tid 1 to smithi01698494-110 2023-01-25T05:22:28.326 INFO:tasks.rados.rados.0.smithi016.stdout:110: finishing write tid 2 to smithi01698494-110 2023-01-25T05:22:28.326 INFO:tasks.rados.rados.0.smithi016.stdout:110: finishing write tid 3 to smithi01698494-110 2023-01-25T05:22:28.326 INFO:tasks.rados.rados.0.smithi016.stdout:109: done (15 left) 2023-01-25T05:22:28.326 INFO:tasks.rados.rados.0.smithi016.stdout:125: write initial oid 125 2023-01-25T05:22:28.327 INFO:tasks.rados.rados.0.smithi016.stdout:125: seq_num 124 ranges {484068=792524,1906473=177595} 2023-01-25T05:22:28.329 INFO:tasks.rados.rados.0.smithi016.stdout:125: writing smithi01698494-125 from 484068 to 1276592 tid 1 2023-01-25T05:22:28.334 INFO:tasks.rados.rados.0.smithi016.stdout:125: writing smithi01698494-125 from 1906473 to 2084068 tid 2 2023-01-25T05:22:28.334 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.334 INFO:tasks.rados.rados.0.smithi016.stdout:110: finishing write tid 4 to smithi01698494-110 2023-01-25T05:22:28.335 INFO:tasks.rados.rados.0.smithi016.stdout:110: finishing write tid 5 to smithi01698494-110 2023-01-25T05:22:28.335 INFO:tasks.rados.rados.0.smithi016.stdout:110: oid 110 updating version 0 to 92 2023-01-25T05:22:28.335 INFO:tasks.rados.rados.0.smithi016.stdout:110: oid 110 updating version 92 to 95 2023-01-25T05:22:28.335 INFO:tasks.rados.rados.0.smithi016.stdout:110: oid 110 version 95 is already newer than 93 2023-01-25T05:22:28.336 INFO:tasks.rados.rados.0.smithi016.stdout:110: oid 110 version 95 is already newer than 94 2023-01-25T05:22:28.336 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 110 v 95 (ObjNum 109 snap 0 seq_num 109) dirty exists 2023-01-25T05:22:28.336 INFO:tasks.rados.rados.0.smithi016.stdout:110: left oid 110 (ObjNum 109 snap 0 seq_num 109) 2023-01-25T05:22:28.336 INFO:tasks.rados.rados.0.smithi016.stdout:111: finishing write tid 1 to smithi01698494-111 2023-01-25T05:22:28.336 INFO:tasks.rados.rados.0.smithi016.stdout:111: finishing write tid 2 to smithi01698494-111 2023-01-25T05:22:28.337 INFO:tasks.rados.rados.0.smithi016.stdout:111: finishing write tid 3 to smithi01698494-111 2023-01-25T05:22:28.337 INFO:tasks.rados.rados.0.smithi016.stdout:111: finishing write tid 4 to smithi01698494-111 2023-01-25T05:22:28.337 INFO:tasks.rados.rados.0.smithi016.stdout:110: done (15 left) 2023-01-25T05:22:28.337 INFO:tasks.rados.rados.0.smithi016.stdout:126: write initial oid 126 2023-01-25T05:22:28.337 INFO:tasks.rados.rados.0.smithi016.stdout:126: seq_num 125 ranges {500875=467773,1564955=535920} 2023-01-25T05:22:28.339 INFO:tasks.rados.rados.0.smithi016.stdout:126: writing smithi01698494-126 from 500875 to 968648 tid 1 2023-01-25T05:22:28.345 INFO:tasks.rados.rados.0.smithi016.stdout:126: writing smithi01698494-126 from 1564955 to 2100875 tid 2 2023-01-25T05:22:28.346 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.346 INFO:tasks.rados.rados.0.smithi016.stdout:111: finishing write tid 5 to smithi01698494-111 2023-01-25T05:22:28.346 INFO:tasks.rados.rados.0.smithi016.stdout:111: oid 111 updating version 0 to 65 2023-01-25T05:22:28.346 INFO:tasks.rados.rados.0.smithi016.stdout:111: oid 111 updating version 65 to 68 2023-01-25T05:22:28.346 INFO:tasks.rados.rados.0.smithi016.stdout:111: oid 111 version 68 is already newer than 66 2023-01-25T05:22:28.347 INFO:tasks.rados.rados.0.smithi016.stdout:111: oid 111 version 68 is already newer than 67 2023-01-25T05:22:28.347 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 111 v 68 (ObjNum 110 snap 0 seq_num 110) dirty exists 2023-01-25T05:22:28.347 INFO:tasks.rados.rados.0.smithi016.stdout:111: left oid 111 (ObjNum 110 snap 0 seq_num 110) 2023-01-25T05:22:28.347 INFO:tasks.rados.rados.0.smithi016.stdout:111: done (15 left) 2023-01-25T05:22:28.347 INFO:tasks.rados.rados.0.smithi016.stdout:127: write initial oid 127 2023-01-25T05:22:28.348 INFO:tasks.rados.rados.0.smithi016.stdout:127: seq_num 126 ranges {517682=543022,1623437=494245} 2023-01-25T05:22:28.351 INFO:tasks.rados.rados.0.smithi016.stdout:127: writing smithi01698494-127 from 517682 to 1060704 tid 1 2023-01-25T05:22:28.358 INFO:tasks.rados.rados.0.smithi016.stdout:127: writing smithi01698494-127 from 1623437 to 2117682 tid 2 2023-01-25T05:22:28.358 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.358 INFO:tasks.rados.rados.0.smithi016.stdout:112: finishing write tid 1 to smithi01698494-112 2023-01-25T05:22:28.358 INFO:tasks.rados.rados.0.smithi016.stdout:112: finishing write tid 2 to smithi01698494-112 2023-01-25T05:22:28.359 INFO:tasks.rados.rados.0.smithi016.stdout:112: finishing write tid 3 to smithi01698494-112 2023-01-25T05:22:28.359 INFO:tasks.rados.rados.0.smithi016.stdout:112: finishing write tid 4 to smithi01698494-112 2023-01-25T05:22:28.359 INFO:tasks.rados.rados.0.smithi016.stdout:112: finishing write tid 5 to smithi01698494-112 2023-01-25T05:22:28.359 INFO:tasks.rados.rados.0.smithi016.stdout:112: oid 112 updating version 0 to 98 2023-01-25T05:22:28.360 INFO:tasks.rados.rados.0.smithi016.stdout:112: oid 112 updating version 98 to 99 2023-01-25T05:22:28.364 INFO:tasks.rados.rados.0.smithi016.stdout:112: oid 112 version 99 is already newer than 96 2023-01-25T05:22:28.364 INFO:tasks.rados.rados.0.smithi016.stdout:112: oid 112 version 99 is already newer than 97 2023-01-25T05:22:28.365 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 112 v 99 (ObjNum 111 snap 0 seq_num 111) dirty exists 2023-01-25T05:22:28.365 INFO:tasks.rados.rados.0.smithi016.stdout:112: left oid 112 (ObjNum 111 snap 0 seq_num 111) 2023-01-25T05:22:28.365 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 1 to smithi01698494-113 2023-01-25T05:22:28.365 INFO:tasks.rados.rados.0.smithi016.stdout:112: done (15 left) 2023-01-25T05:22:28.365 INFO:tasks.rados.rados.0.smithi016.stdout:128: write initial oid 128 2023-01-25T05:22:28.366 INFO:tasks.rados.rados.0.smithi016.stdout:128: seq_num 127 ranges {534489=618271,1565566=568923} 2023-01-25T05:22:28.366 INFO:tasks.rados.rados.0.smithi016.stdout:128: writing smithi01698494-128 from 534489 to 1152760 tid 1 2023-01-25T05:22:28.370 INFO:tasks.rados.rados.0.smithi016.stdout:128: writing smithi01698494-128 from 1565566 to 2134489 tid 2 2023-01-25T05:22:28.371 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.371 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 2 to smithi01698494-113 2023-01-25T05:22:28.371 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 3 to smithi01698494-113 2023-01-25T05:22:28.371 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 4 to smithi01698494-113 2023-01-25T05:22:28.372 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 5 to smithi01698494-113 2023-01-25T05:22:28.372 INFO:tasks.rados.rados.0.smithi016.stdout:113: finishing write tid 6 to smithi01698494-113 2023-01-25T05:22:28.372 INFO:tasks.rados.rados.0.smithi016.stdout:113: oid 113 updating version 0 to 103 2023-01-25T05:22:28.372 INFO:tasks.rados.rados.0.smithi016.stdout:113: oid 113 version 103 is already newer than 100 2023-01-25T05:22:28.372 INFO:tasks.rados.rados.0.smithi016.stdout:113: oid 113 version 103 is already newer than 101 2023-01-25T05:22:28.373 INFO:tasks.rados.rados.0.smithi016.stdout:113: oid 113 updating version 103 to 104 2023-01-25T05:22:28.373 INFO:tasks.rados.rados.0.smithi016.stdout:113: oid 113 version 104 is already newer than 102 2023-01-25T05:22:28.373 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 113 v 104 (ObjNum 112 snap 0 seq_num 112) dirty exists 2023-01-25T05:22:28.373 INFO:tasks.rados.rados.0.smithi016.stdout:113: left oid 113 (ObjNum 112 snap 0 seq_num 112) 2023-01-25T05:22:28.373 INFO:tasks.rados.rados.0.smithi016.stdout:114: finishing write tid 1 to smithi01698494-114 2023-01-25T05:22:28.374 INFO:tasks.rados.rados.0.smithi016.stdout:113: done (15 left) 2023-01-25T05:22:28.374 INFO:tasks.rados.rados.0.smithi016.stdout:129: write initial oid 129 2023-01-25T05:22:28.374 INFO:tasks.rados.rados.0.smithi016.stdout:129: seq_num 128 ranges {551296=693520,2024048=127248} 2023-01-25T05:22:28.376 INFO:tasks.rados.rados.0.smithi016.stdout:129: writing smithi01698494-129 from 551296 to 1244816 tid 1 2023-01-25T05:22:28.383 INFO:tasks.rados.rados.0.smithi016.stdout:129: writing smithi01698494-129 from 2024048 to 2151296 tid 2 2023-01-25T05:22:28.383 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.383 INFO:tasks.rados.rados.0.smithi016.stdout:114: finishing write tid 2 to smithi01698494-114 2023-01-25T05:22:28.383 INFO:tasks.rados.rados.0.smithi016.stdout:114: finishing write tid 3 to smithi01698494-114 2023-01-25T05:22:28.383 INFO:tasks.rados.rados.0.smithi016.stdout:114: finishing write tid 4 to smithi01698494-114 2023-01-25T05:22:28.384 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 1 to smithi01698494-115 2023-01-25T05:22:28.384 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 2 to smithi01698494-115 2023-01-25T05:22:28.384 INFO:tasks.rados.rados.0.smithi016.stdout:114: finishing write tid 5 to smithi01698494-114 2023-01-25T05:22:28.384 INFO:tasks.rados.rados.0.smithi016.stdout:114: oid 114 updating version 0 to 67 2023-01-25T05:22:28.385 INFO:tasks.rados.rados.0.smithi016.stdout:114: oid 114 version 67 is already newer than 66 2023-01-25T05:22:28.385 INFO:tasks.rados.rados.0.smithi016.stdout:114: oid 114 updating version 67 to 68 2023-01-25T05:22:28.385 INFO:tasks.rados.rados.0.smithi016.stdout:114: oid 114 updating version 68 to 69 2023-01-25T05:22:28.385 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 114 v 69 (ObjNum 113 snap 0 seq_num 113) dirty exists 2023-01-25T05:22:28.385 INFO:tasks.rados.rados.0.smithi016.stdout:114: left oid 114 (ObjNum 113 snap 0 seq_num 113) 2023-01-25T05:22:28.386 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 3 to smithi01698494-115 2023-01-25T05:22:28.386 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 4 to smithi01698494-115 2023-01-25T05:22:28.386 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 5 to smithi01698494-115 2023-01-25T05:22:28.386 INFO:tasks.rados.rados.0.smithi016.stdout:115: finishing write tid 6 to smithi01698494-115 2023-01-25T05:22:28.386 INFO:tasks.rados.rados.0.smithi016.stdout:115: oid 115 updating version 0 to 61 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:115: oid 115 version 61 is already newer than 60 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:115: oid 115 updating version 61 to 63 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:115: oid 115 updating version 63 to 64 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:115: oid 115 version 64 is already newer than 62 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 115 v 64 (ObjNum 114 snap 0 seq_num 114) dirty exists 2023-01-25T05:22:28.387 INFO:tasks.rados.rados.0.smithi016.stdout:115: left oid 115 (ObjNum 114 snap 0 seq_num 114) 2023-01-25T05:22:28.388 INFO:tasks.rados.rados.0.smithi016.stdout:116: finishing write tid 1 to smithi01698494-116 2023-01-25T05:22:28.388 INFO:tasks.rados.rados.0.smithi016.stdout:114: done (15 left) 2023-01-25T05:22:28.388 INFO:tasks.rados.rados.0.smithi016.stdout:115: done (14 left) 2023-01-25T05:22:28.388 INFO:tasks.rados.rados.0.smithi016.stdout:130: write initial oid 130 2023-01-25T05:22:28.389 INFO:tasks.rados.rados.0.smithi016.stdout:130: seq_num 129 ranges {568103=768769,2082530=85573} 2023-01-25T05:22:28.389 INFO:tasks.rados.rados.0.smithi016.stdout:130: writing smithi01698494-130 from 568103 to 1336872 tid 1 2023-01-25T05:22:28.394 INFO:tasks.rados.rados.0.smithi016.stdout:130: writing smithi01698494-130 from 2082530 to 2168103 tid 2 2023-01-25T05:22:28.394 INFO:tasks.rados.rados.0.smithi016.stdout:131: write initial oid 131 2023-01-25T05:22:28.394 INFO:tasks.rados.rados.0.smithi016.stdout:131: seq_num 130 ranges {584910=560371,1857365=327545} 2023-01-25T05:22:28.400 INFO:tasks.rados.rados.0.smithi016.stdout:131: writing smithi01698494-131 from 584910 to 1145281 tid 1 2023-01-25T05:22:28.407 INFO:tasks.rados.rados.0.smithi016.stdout:131: writing smithi01698494-131 from 1857365 to 2184910 tid 2 2023-01-25T05:22:28.407 INFO:tasks.rados.rados.0.smithi016.stdout: waiting on 16 2023-01-25T05:22:28.407 INFO:tasks.rados.rados.0.smithi016.stdout:116: finishing write tid 2 to smithi01698494-116 2023-01-25T05:22:28.408 INFO:tasks.rados.rados.0.smithi016.stdout:116: finishing write tid 3 to smithi01698494-116 2023-01-25T05:22:28.408 INFO:tasks.rados.rados.0.smithi016.stdout:116: finishing write tid 4 to smithi01698494-116 2023-01-25T05:22:28.408 INFO:tasks.rados.rados.0.smithi016.stdout:116: finishing write tid 5 to smithi01698494-116 2023-01-25T05:22:28.408 INFO:tasks.rados.rados.0.smithi016.stdout:116: oid 116 updating version 0 to 108 2023-01-25T05:22:28.409 INFO:tasks.rados.rados.0.smithi016.stdout:116: oid 116 version 108 is already newer than 105 2023-01-25T05:22:28.409 INFO:tasks.rados.rados.0.smithi016.stdout:116: oid 116 version 108 is already newer than 106 2023-01-25T05:22:28.409 INFO:tasks.rados.rados.0.smithi016.stdout:116: oid 116 version 108 is already newer than 107 2023-01-25T05:22:28.409 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 116 v 108 (ObjNum 115 snap 0 seq_num 115) dirty exists 2023-01-25T05:22:28.409 INFO:tasks.rados.rados.0.smithi016.stdout:116: left oid 116 (ObjNum 115 snap 0 seq_num 115) 2023-01-25T05:22:28.410 INFO:tasks.rados.rados.0.smithi016.stdout:117: finishing write tid 1 to smithi01698494-117 2023-01-25T05:22:28.410 INFO:tasks.rados.rados.0.smithi016.stdout:117: finishing write tid 2 to smithi01698494-117 2023-01-25T05:22:28.410 INFO:tasks.rados.rados.0.smithi016.stdout:117: finishing write tid 3 to smithi01698494-117 2023-01-25T05:22:28.410 INFO:tasks.rados.rados.0.smithi016.stdout:117: finishing write tid 4 to smithi01698494-117 2023-01-25T05:22:28.410 INFO:tasks.rados.rados.0.smithi016.stdout:117: finishing write tid 5 to smithi01698494-117 2023-01-25T05:22:28.411 INFO:tasks.rados.rados.0.smithi016.stdout:117: oid 117 updating version 0 to 67 2023-01-25T05:22:28.411 INFO:tasks.rados.rados.0.smithi016.stdout:117: oid 117 version 67 is already newer than 65 2023-01-25T05:22:28.411 INFO:tasks.rados.rados.0.smithi016.stdout:117: oid 117 version 67 is already newer than 62 2023-01-25T05:22:28.411 INFO:tasks.rados.rados.0.smithi016.stdout:117: oid 117 version 67 is already newer than 66 2023-01-25T05:22:28.411 INFO:tasks.rados.rados.0.smithi016.stdout:update_object_version oid 117 v 67 (ObjNum 116 snap 0 seq_num 116) dirty exists 2023-01-25T05:22:28.412 INFO:tasks.rados.rados.0.smithi016.stdout:117: left oid 117 (ObjNum 116 snap 0 seq_num 116) 2023-01-25T05:22:28.412 INFO:tasks.rados.rados.0.smithi016.stdout:118: finishing write tid 1 to smithi01698494-118 2023-01-25T05:22:28.412 INFO:tasks.rados.rados.0.smithi016.stdout:118: finishing write tid 2 to smithi01698494-118 2023-01-25T05:22:28.412 INFO:tasks.rados.rados.0.smithi016.stdout:118: finishing write tid 3 to smithi01698494-118 2023-01-25T05:22:28.412 INFO:tasks.rados.rados.0.smithi016.stdout:116: done (15 left) 2023-01-25T05:22:28.413 INFO:tasks.rados.rados.0.smithi016.stdout:117: done (14 left) 2023-01-25T05:22:28.413 INFO:tasks.rados.rados.0.smithi016.stdout:132: write initial oid 132 2023-01-25T05:22:28.413 INFO:tasks.rados.rados.0.smithi016.stdout:132: seq_num 131 ranges {601717=635620,1799494=402223} 2023-01-25T05:22:28.413 INFO:tasks.rados.rados.0.smithi016.stdout:132: writing smithi01698494-132 from 601717 to 1237337 tid 1 2023-01-25T05:22:28.418 INFO:tasks.rados.rados.0.smithi016.stdout:132: writing smithi01698494-132 from 1799494 to 2201717 tid 2 2023-01-25T05:22:28.418 INFO:tasks.rados.rados.0.smithi016.stdout:133: write initial oid 133 2023-01-25T05:22:28.419 INFO:tasks.rados.rados.0.smithi016.stdout:133: seq_num 132 ranges {618524=710869,1857976=360548} 2023-01-25T05:22:28.425 INFO:tasks.rados.rados.0.smithi016.stdout:133: writing smithi01698494-133 from 618524 to 1329393 tid 1 2023-01-25T05:22:28.432 INFO:tasks.rados.rados.0.smithi016.stdout:133: writing smithi01698494-133