2023-01-25T12:17:08.168 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T12:17:08.169 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420 --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} --name yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420/orig.config.yaml 2023-01-25T12:17:08.223 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T12:17:08.295 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420 branch: wip-yuri6-testing-2023-01-24-0746 description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} email: yweinstein@radhat.com first_in_suite: false job_id: '7137420' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri6-testing-2023-01-24-0746 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri6-testing-2023-01-24-0746 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri6-testing-2023-01-24-0746 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c targets: smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfk+bkc/COlUouKAKfgLpBP1ZRmF2gypxXp8/D5fjmSlri4t++5qHNM2KPmv/HoJ+k+odgVbn5Dtb8U+B3cvDrT907m5x4vQhQ6G51IdlrEdMlofDgC6zdYCp310WTOjvl51THqr4r5tUtGLcSTj6MWn35+FxhXvJeBs/4fqwEVjgS9JeRQvvWodXSRTECwRAeY6T+EhrsQJl6W7UuM4OH3KXZfygeNuNBjhKmR3dQjMSuIu6l0G2lHKTPE9ih1KCw9fa+3htlPqgiJ06qMfAvg+nIAo9GT92GPAi8Nc+2ofuETous3Q8MTxBI6EPnx0OnHgEJ4jHqNXcXzfDFMazILe6hbDlXvIHgiKvSSEVJxFkPXW2K7mODtN4UnsPxlQ02ix36iC7VA4NR665EHsg2YJm8TcoUhwVqQW8VBaX70W2C6LMLNdffj0fjjhiOdeOtCuU1i60KBTXX/HVjGUqU6U43I7bKMv7OVr8ogscxeAQgx96GynY82qHPqt9I9xM= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOO/Pj+7iKcD4i8uJKpXzqNEfrKvFYOp8XUDl6AcwxPQswLjdAFYX5fXlkB4GsKQJi6YyM6TX6ejJ1mZ2SeJzPc= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache readproxy - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:27:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T12:17:08.295 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa; will attempt to use it 2023-01-25T12:17:08.296 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa/tasks 2023-01-25T12:17:08.296 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T12:17:08.298 INFO:teuthology.task.internal:Checking packages... 2023-01-25T12:17:08.332 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44e4095da12a8269eb6cfe163cbb0d00b1a6412c' 2023-01-25T12:17:08.332 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T12:17:08.333 INFO:teuthology.packaging:ref: None 2023-01-25T12:17:08.333 INFO:teuthology.packaging:tag: None 2023-01-25T12:17:08.333 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-01-24-0746 2023-01-25T12:17:08.334 INFO:teuthology.packaging:sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:17:08.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2023-01-24-0746 2023-01-25T12:17:08.512 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1986.g44e4095d 2023-01-25T12:17:08.512 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T12:17:08.519 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T12:17:08.519 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T12:17:08.543 INFO:teuthology.task.internal:Saving configuration 2023-01-25T12:17:08.562 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T12:17:08.568 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T12:17:08.594 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 12:12:03.975766', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPH7UHcK0tpV+uev3xAmKrCb0V122xdgRb2I/YMpiCLCh66LD2Dr340B8mRjyJUo4IeI3Zpm1Scwz+GZk63uttI='} 2023-01-25T12:17:08.616 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 12:12:03.974106', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOO/Pj+7iKcD4i8uJKpXzqNEfrKvFYOp8XUDl6AcwxPQswLjdAFYX5fXlkB4GsKQJi6YyM6TX6ejJ1mZ2SeJzPc='} 2023-01-25T12:17:08.616 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T12:17:08.622 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T12:17:08.623 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T12:17:08.623 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T12:17:08.693 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19aab4a670>, signals=[15]) 2023-01-25T12:17:08.695 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T12:17:08.704 INFO:teuthology.task.internal:Opening connections... 2023-01-25T12:17:08.705 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:17:08.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:17:08.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:17:08.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:17:08.887 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T12:17:08.894 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2023-01-25T12:17:08.915 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2023-01-25T12:17:08.916 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-01-25T12:17:08.977 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2023-01-25T12:17:08.977 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8" 2023-01-25T12:17:08.978 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2023-01-25T12:17:08.978 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2023-01-25T12:17:08.979 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8" 2023-01-25T12:17:08.979 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2023-01-25T12:17:08.979 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T12:17:08.979 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-01-25T12:17:08.980 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T12:17:08.980 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2023-01-25T12:17:08.980 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T12:17:08.980 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T12:17:08.980 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T12:17:08.981 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-01-25T12:17:08.996 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-01-25T12:17:09.016 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-01-25T12:17:09.016 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-01-25T12:17:09.074 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2023-01-25T12:17:09.074 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8" 2023-01-25T12:17:09.075 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2023-01-25T12:17:09.075 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2023-01-25T12:17:09.075 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8" 2023-01-25T12:17:09.075 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-01-25T12:17:09.075 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T12:17:09.076 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-01-25T12:17:09.076 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T12:17:09.076 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2023-01-25T12:17:09.076 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T12:17:09.076 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T12:17:09.077 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T12:17:09.077 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-01-25T12:17:09.092 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T12:17:09.106 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T12:17:09.115 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T12:17:09.116 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2023-01-25T12:17:09.119 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2023-01-25T12:17:09.139 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T12:17:09.147 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T12:17:09.148 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2023-01-25T12:17:09.181 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2023-01-25T12:17:09.209 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T12:17:09.261 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T12:17:09.283 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T12:17:09.284 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T12:17:09.285 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T12:17:09.286 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T12:17:09.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:17:09.304 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2023-01-25T12:17:09.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:17:09.307 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-01-25T12:17:09.362 INFO:teuthology.orchestra.run.smithi087.stdout:4.18.0-408.el8.x86_64 2023-01-25T12:17:09.362 INFO:teuthology.task.kernel:Running kernel on smithi087: 4.18.0-408.el8.x86_64 2023-01-25T12:17:09.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2023-01-25T12:17:09.369 INFO:teuthology.orchestra.run.smithi067.stdout:4.18.0-408.el8.x86_64 2023-01-25T12:17:09.369 INFO:teuthology.task.kernel:Running kernel on smithi067: 4.18.0-408.el8.x86_64 2023-01-25T12:17:09.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2023-01-25T12:17:12.943 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - AppStream 32 MB/s | 27 MB 00:00 2023-01-25T12:17:13.090 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - AppStream 30 MB/s | 27 MB 00:00 2023-01-25T12:17:18.790 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - BaseOS 37 MB/s | 27 MB 00:00 2023-01-25T12:17:18.943 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - BaseOS 39 MB/s | 27 MB 00:00 2023-01-25T12:17:23.064 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - Extras 141 kB/s | 18 kB 00:00 2023-01-25T12:17:23.384 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - Extras 169 kB/s | 18 kB 00:00 2023-01-25T12:17:23.458 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T12:17:23.769 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - PowerTools 23 MB/s | 5.6 MB 00:00 2023-01-25T12:17:24.674 INFO:teuthology.orchestra.run.smithi087.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-01-25T12:17:25.060 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-01-25T12:17:26.607 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 13 MB 00:01 2023-01-25T12:17:26.861 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 13 MB 00:01 2023-01-25T12:17:29.841 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 480 kB/s | 24 kB 00:00 2023-01-25T12:17:30.386 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-01-25T12:17:32.947 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T12:17:32.948 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T12:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:17:33.101 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:17:33.102 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T12:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T12:17:33.103 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T12:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:17:33.104 INFO:teuthology.orchestra.run.smithi087.stdout:Install 3 Packages 2023-01-25T12:17:33.105 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 85 M 2023-01-25T12:17:33.106 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 94 M 2023-01-25T12:17:33.107 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:17:33.409 INFO:teuthology.orchestra.run.smithi087.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 30 MB/s | 9.2 MB 00:00 2023-01-25T12:17:33.659 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T12:17:33.660 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T12:17:33.834 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:17:33.835 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:17:33.835 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:17:33.835 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:17:33.836 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:17:33.836 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T12:17:33.836 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T12:17:33.836 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T12:17:33.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:17:33.837 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:17:33.837 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:17:33.837 INFO:teuthology.orchestra.run.smithi067.stdout:Install 3 Packages 2023-01-25T12:17:33.837 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:17:33.838 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 85 M 2023-01-25T12:17:33.838 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 94 M 2023-01-25T12:17:33.838 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:17:34.226 INFO:teuthology.orchestra.run.smithi087.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 37 MB/s | 41 MB 00:01 2023-01-25T12:17:34.287 INFO:teuthology.orchestra.run.smithi067.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 20 MB/s | 9.2 MB 00:00 2023-01-25T12:17:34.752 INFO:teuthology.orchestra.run.smithi087.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-01-25T12:17:34.755 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:34.755 INFO:teuthology.orchestra.run.smithi087.stdout:Total 51 MB/s | 85 MB 00:01 2023-01-25T12:17:34.971 INFO:teuthology.orchestra.run.smithi067.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 36 MB/s | 41 MB 00:01 2023-01-25T12:17:35.388 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:17:35.429 INFO:teuthology.orchestra.run.smithi067.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-01-25T12:17:35.432 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:35.432 INFO:teuthology.orchestra.run.smithi067.stdout:Total 53 MB/s | 85 MB 00:01 2023-01-25T12:17:35.803 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:17:35.803 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:17:36.099 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:17:36.509 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:17:36.509 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:17:37.884 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:17:37.909 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:17:39.693 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:17:39.719 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:17:40.650 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:17:41.243 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:17:42.581 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:17:43.226 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:17:45.033 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:17:45.451 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:17:47.159 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:17:48.775 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:17:49.067 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:17:51.136 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:17:51.378 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:10.966 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:10.992 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:11.570 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:11.571 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:18:11.571 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:18:12.149 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:12.149 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:18:12.149 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:18:12.149 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T12:18:12.150 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T12:18:12.150 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:18:12.150 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:18:12.286 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel --last | head -n 1 2023-01-25T12:18:12.485 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 12:17:48 PM UTC 2023-01-25T12:18:12.486 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T12:18:12.487 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T12:18:12.487 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T12:18:12.487 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T12:18:12.487 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-01-25T12:18:12.488 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T12:18:12.488 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel --last | head -n 1 2023-01-25T12:18:12.675 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 12:17:48 PM UTC 2023-01-25T12:18:12.676 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T12:18:12.677 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2023-01-25T12:18:12.709 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2023-01-25T12:18:12.710 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2023-01-25T12:18:12.710 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.02 2023-01-25T12:18:12.710 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 129.el8 2023-01-25T12:18:12.711 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2023-01-25T12:18:12.711 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T12:18:12.711 INFO:teuthology.orchestra.run.smithi087.stdout:Group : System Environment/Base 2023-01-25T12:18:12.712 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 9265586 2023-01-25T12:18:12.712 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2023-01-25T12:18:12.712 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T12:18:12.712 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T12:18:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T12:18:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T12:18:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:Relocations : (not relocatable) 2023-01-25T12:18:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : CentOS BuildSystem 2023-01-25T12:18:12.713 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2023-01-25T12:18:12.714 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T12:18:12.714 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2023-01-25T12:18:12.714 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2023-01-25T12:18:12.714 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:18:12.714 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T12:18:12.715 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T12:18:12.715 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T12:18:12.715 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2023-01-25T12:18:12.716 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:18:12.716 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T12:18:12.717 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T12:18:12.717 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 4.18.0-448.el8.x86_64 2023-01-25T12:18:12.718 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T12:18:14.114 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2023-01-25T12:18:14.786 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:14.813 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T12:18:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T12:18:15.927 INFO:teuthology.orchestra.run.smithi087.stderr:done 2023-01-25T12:18:15.929 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2023-01-25T12:18:15.959 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.MDeAm2eyCc 2023-01-25T12:18:15.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MDeAm2eyCc 2023-01-25T12:18:16.033 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.MDeAm2eyCc 2023-01-25T12:18:16.180 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.MDeAm2eyCc is 6KB 2023-01-25T12:18:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T12:18:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:18:16.222 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:18:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T12:18:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T12:18:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:18:16.223 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:18:16.228 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.MDeAm2eyCc 2023-01-25T12:18:16.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T12:18:16.309 INFO:teuthology.orchestra.run.smithi087.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T12:18:16.309 INFO:teuthology.orchestra.run.smithi087.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T12:18:16.310 INFO:teuthology.orchestra.run.smithi087.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T12:18:16.310 INFO:teuthology.orchestra.run.smithi087.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T12:18:16.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T12:18:16.337 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel --last | head -n 1 2023-01-25T12:18:16.471 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2023-01-25T12:18:16.540 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 12:17:51 PM UTC 2023-01-25T12:18:16.542 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T12:18:16.543 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T12:18:16.543 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T12:18:16.543 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T12:18:16.543 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-25T12:18:16.543 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T12:18:16.544 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel --last | head -n 1 2023-01-25T12:18:16.739 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 12:17:51 PM UTC 2023-01-25T12:18:16.741 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T12:18:16.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2023-01-25T12:18:16.776 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2023-01-25T12:18:16.776 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2023-01-25T12:18:16.776 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.02 2023-01-25T12:18:16.777 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 129.el8 2023-01-25T12:18:16.777 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2023-01-25T12:18:16.777 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T12:18:16.777 INFO:teuthology.orchestra.run.smithi067.stdout:Group : System Environment/Base 2023-01-25T12:18:16.777 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 9265586 2023-01-25T12:18:16.778 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2023-01-25T12:18:16.778 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T12:18:16.778 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T12:18:16.778 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T12:18:16.778 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T12:18:16.779 INFO:teuthology.orchestra.run.smithi067.stdout:Relocations : (not relocatable) 2023-01-25T12:18:16.779 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : CentOS BuildSystem 2023-01-25T12:18:16.779 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2023-01-25T12:18:16.779 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T12:18:16.779 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2023-01-25T12:18:16.780 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2023-01-25T12:18:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:18:16.780 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T12:18:16.780 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T12:18:16.780 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T12:18:16.781 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2023-01-25T12:18:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:18:16.781 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T12:18:16.782 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T12:18:16.783 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 4.18.0-448.el8.x86_64 2023-01-25T12:18:16.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T12:18:18.091 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2023-01-25T12:18:19.791 INFO:teuthology.orchestra.run.smithi067.stderr:done 2023-01-25T12:18:19.792 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-01-25T12:18:19.813 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.JUuf5CPe87 2023-01-25T12:18:19.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JUuf5CPe87 2023-01-25T12:18:19.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.JUuf5CPe87 2023-01-25T12:18:20.034 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.JUuf5CPe87 is 6KB 2023-01-25T12:18:20.082 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.JUuf5CPe87 2023-01-25T12:18:20.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T12:18:20.165 INFO:teuthology.orchestra.run.smithi067.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T12:18:20.165 INFO:teuthology.orchestra.run.smithi067.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T12:18:20.165 INFO:teuthology.orchestra.run.smithi067.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T12:18:20.166 INFO:teuthology.orchestra.run.smithi067.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T12:18:20.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T12:18:20.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2023-01-25T12:18:46.496 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-01-25T12:18:46.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-01-25T12:18:46.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:18:50.316 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-25T12:18:50.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-01-25T12:18:50.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:19:24.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-01-25T12:19:46.520 DEBUG:teuthology.orchestra.remote:timed out 2023-01-25T12:19:54.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-01-25T12:19:54.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:20:02.273 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-01-25T12:20:02.634 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-01-25T12:20:02.634 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-25T12:20:02.635 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-01-25T12:20:02.653 INFO:teuthology.orchestra.run.smithi067.stdout:4.18.0-448.el8.x86_64 2023-01-25T12:20:02.654 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T12:20:02.654 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T12:20:02.654 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T12:20:03.655 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T12:20:03.656 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T12:20:03.751 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2023-01-25T12:20:03.780 DEBUG:teuthology.parallel:result is None 2023-01-25T12:20:16.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-01-25T12:20:16.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:20:16.873 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-01-25T12:20:17.139 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-01-25T12:20:17.140 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-01-25T12:20:17.140 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2023-01-25T12:20:17.199 INFO:teuthology.orchestra.run.smithi087.stdout:4.18.0-448.el8.x86_64 2023-01-25T12:20:17.199 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T12:20:17.199 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T12:20:17.200 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T12:20:18.200 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T12:20:18.201 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T12:20:18.321 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2023-01-25T12:20:18.335 DEBUG:teuthology.parallel:result is None 2023-01-25T12:20:18.336 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T12:20:18.342 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T12:20:18.343 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T12:20:18.347 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T12:20:18.368 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T12:20:18.375 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T12:20:18.382 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T12:20:18.383 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T12:20:18.411 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T12:20:18.445 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T12:20:18.455 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T12:20:18.455 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T12:20:18.477 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T12:20:18.520 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:18.527 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:18.532 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:18.538 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:18.540 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T12:20:18.547 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T12:20:18.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T12:20:18.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T12:20:18.605 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T12:20:18.614 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T12:20:18.615 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T12:20:18.621 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T12:20:18.664 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:20:18.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:20:18.775 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:20:18.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:20:18.903 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:20:18.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T12:20:18.971 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:20:19.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:20:19.071 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:20:19.156 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:20:19.202 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:20:19.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T12:20:19.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2023-01-25T12:20:19.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2023-01-25T12:20:19.335 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T12:20:19.364 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T12:20:19.659 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T12:20:19.667 INFO:teuthology.task.internal:Starting timer... 2023-01-25T12:20:19.667 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T12:20:19.680 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T12:20:19.690 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T12:20:19.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2023-01-25T12:20:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2023-01-25T12:20:19.766 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2023-01-25T12:20:19.839 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2023-01-25T12:20:19.841 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T12:20:19.841 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2023-01-25T12:20:19.870 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2023-01-25T12:20:19.871 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2023-01-25T12:20:19.904 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2023-01-25T12:20:19.904 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi087.front.sepia.ceph.com': 'permissive'} 2023-01-25T12:20:19.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T12:20:19.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:20:19.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T12:20:19.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:20:19.972 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T12:20:19.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2023-01-25T12:20:20.008 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2023-01-25T12:20:20.049 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T12:20:20.060 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T12:20:20.060 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T12:20:20.078 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T12:20:20.079 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi087.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T12:26:51.926 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com')] 2023-01-25T12:26:51.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-01-25T12:26:51.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:26:52.000 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-01-25T12:26:52.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-01-25T12:26:52.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:26:52.136 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-01-25T12:26:52.207 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T12:26:52.217 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T12:26:52.218 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T12:26:52.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T12:26:52.221 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T12:26:52.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T12:26:52.251 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T12:26:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T12:26:52.279 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T12:26:52.293 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T12:26:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2023-01-25T12:26:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2023-01-25T12:26:52.366 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2023-01-25T12:26:52.371 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T12:26:52.377 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2023-01-25T12:26:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T12:26:52.391 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T12:26:52.405 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T12:26:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2023-01-25T12:26:52.466 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T12:26:52.466 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-01-25T12:26:52.466 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.467 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.467 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.467 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.468 INFO:teuthology.orchestra.run.smithi087.stderr:bash: ntpq: command not found 2023-01-25T12:26:52.472 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T12:26:52.472 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2023-01-25T12:26:52.472 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.473 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.473 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.473 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:26:52.474 INFO:teuthology.run_tasks:Running task install... 2023-01-25T12:26:52.485 DEBUG:teuthology.task.install:project ceph 2023-01-25T12:26:52.485 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T12:26:52.486 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'} 2023-01-25T12:26:52.486 INFO:teuthology.task.install:Using flavor: default 2023-01-25T12:26:52.492 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T12:26:52.492 INFO:teuthology.task.install:extra packages: [] 2023-01-25T12:26:52.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo lsb_release -is 2023-01-25T12:26:52.544 INFO:teuthology.orchestra.run.smithi067.stdout:CentOSStream 2023-01-25T12:26:52.546 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:26:52.546 DEBUG:teuthology.orchestra.run.smithi087:> sudo lsb_release -is 2023-01-25T12:26:52.547 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T12:26:52.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo lsb_release -is 2023-01-25T12:26:52.584 INFO:teuthology.orchestra.run.smithi087.stdout:CentOSStream 2023-01-25T12:26:52.585 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:26:52.586 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T12:26:52.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo lsb_release -is 2023-01-25T12:26:52.623 INFO:teuthology.orchestra.run.smithi067.stdout:CentOSStream 2023-01-25T12:26:52.625 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:26:52.625 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:26:52.663 INFO:teuthology.orchestra.run.smithi087.stdout:CentOSStream 2023-01-25T12:26:52.665 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:26:52.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:26:52.780 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T12:26:52.780 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T12:26:52.815 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T12:26:52.816 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T12:26:52.895 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T12:26:52.895 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:26:52.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T12:26:52.925 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T12:26:52.925 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T12:26:52.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T12:26:52.932 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:26:52.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T12:26:52.960 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T12:26:52.960 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T12:26:52.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T12:26:53.027 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T12:26:53.072 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T12:26:53.105 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T12:26:53.140 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2023-01-25T12:26:53.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2023-01-25T12:26:53.172 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2023-01-25T12:26:53.173 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2023-01-25T12:26:53.408 INFO:teuthology.orchestra.run.smithi067.stdout:61 files removed 2023-01-25T12:26:53.429 INFO:teuthology.orchestra.run.smithi087.stdout:61 files removed 2023-01-25T12:26:53.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw 2023-01-25T12:26:53.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw 2023-01-25T12:26:54.141 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 286 kB/s | 79 kB 00:00 2023-01-25T12:26:54.146 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 287 kB/s | 79 kB 00:00 2023-01-25T12:26:54.584 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 23 kB/s | 9.3 kB 00:00 2023-01-25T12:26:54.588 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 23 kB/s | 9.3 kB 00:00 2023-01-25T12:26:55.014 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-01-25T12:26:55.059 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 3.3 kB/s | 1.5 kB 00:00 2023-01-25T12:26:55.725 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - AppStream 40 MB/s | 27 MB 00:00 2023-01-25T12:26:55.903 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - AppStream 34 MB/s | 27 MB 00:00 2023-01-25T12:27:01.351 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - BaseOS 45 MB/s | 27 MB 00:00 2023-01-25T12:27:01.980 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - BaseOS 39 MB/s | 27 MB 00:00 2023-01-25T12:27:05.591 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - Extras 149 kB/s | 18 kB 00:00 2023-01-25T12:27:06.002 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-25T12:27:06.536 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - Extras 119 kB/s | 18 kB 00:00 2023-01-25T12:27:06.980 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-8 - PowerTools 19 MB/s | 5.6 MB 00:00 2023-01-25T12:27:07.209 INFO:teuthology.orchestra.run.smithi087.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-01-25T12:27:08.254 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T12:27:08.268 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T12:27:09.380 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T12:27:11.843 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 69 kB/s | 24 kB 00:00 2023-01-25T12:27:12.823 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 409 kB/s | 24 kB 00:00 2023-01-25T12:27:15.252 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:27:15.255 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:15.255 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:27:15.255 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:15.255 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:27:15.256 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T12:27:15.256 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2023-01-25T12:27:15.257 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:27:15.257 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:27:15.257 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:27:15.258 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T12:27:15.258 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T12:27:15.259 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T12:27:15.259 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T12:27:15.259 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T12:27:15.260 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T12:27:15.260 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T12:27:15.261 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T12:27:15.261 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T12:27:15.261 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T12:27:15.261 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T12:27:15.262 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T12:27:15.262 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse 2023-01-25T12:27:15.262 INFO:teuthology.orchestra.run.smithi087.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T12:27:15.262 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T12:27:15.263 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T12:27:15.263 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T12:27:15.263 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T12:27:15.263 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T12:27:15.263 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T12:27:15.264 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T12:27:15.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:15.264 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:27:15.264 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:15.264 INFO:teuthology.orchestra.run.smithi087.stdout:Install 21 Packages 2023-01-25T12:27:15.265 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 2 Packages 2023-01-25T12:27:15.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:15.265 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 58 M 2023-01-25T12:27:15.266 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:27:15.895 INFO:teuthology.orchestra.run.smithi087.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 7.6 MB/s | 4.8 MB 00:00 2023-01-25T12:27:15.969 INFO:teuthology.orchestra.run.smithi087.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 325 kB/s | 24 kB 00:00 2023-01-25T12:27:16.078 INFO:teuthology.orchestra.run.smithi087.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 6.9 MB/s | 767 kB 00:00 2023-01-25T12:27:16.178 INFO:teuthology.orchestra.run.smithi087.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 4.6 MB/s | 474 kB 00:00 2023-01-25T12:27:16.508 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:27:16.511 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:27:16.512 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:16.512 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:27:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T12:27:16.512 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2023-01-25T12:27:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:27:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:27:16.513 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:27:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T12:27:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T12:27:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T12:27:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T12:27:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T12:27:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T12:27:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T12:27:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T12:27:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T12:27:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T12:27:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T12:27:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T12:27:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse 2023-01-25T12:27:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T12:27:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T12:27:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T12:27:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T12:27:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T12:27:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T12:27:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T12:27:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T12:27:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:16.517 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:27:16.518 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:16.518 INFO:teuthology.orchestra.run.smithi067.stdout:Install 21 Packages 2023-01-25T12:27:16.518 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 2 Packages 2023-01-25T12:27:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:16.519 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 58 M 2023-01-25T12:27:16.519 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:27:17.004 INFO:teuthology.orchestra.run.smithi087.stdout:(5/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8. 7.5 MB/s | 13 MB 00:01 2023-01-25T12:27:17.089 INFO:teuthology.orchestra.run.smithi087.stdout:(6/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 7.1 MB/s | 6.5 MB 00:00 2023-01-25T12:27:17.113 INFO:teuthology.orchestra.run.smithi087.stdout:(7/23): python3-ceph-argparse-18.0.0-1986.g44e4 411 kB/s | 45 kB 00:00 2023-01-25T12:27:17.188 INFO:teuthology.orchestra.run.smithi087.stdout:(8/23): python3-ceph-common-18.0.0-1986.g44e409 1.1 MB/s | 115 kB 00:00 2023-01-25T12:27:17.221 INFO:teuthology.orchestra.run.smithi087.stdout:(9/23): python3-cephfs-18.0.0-1986.g44e4095d.el 1.7 MB/s | 190 kB 00:00 2023-01-25T12:27:17.480 INFO:teuthology.orchestra.run.smithi087.stdout:(10/23): ceph-common-18.0.0-1986.g44e4095d.el8. 9.7 MB/s | 21 MB 00:02 2023-01-25T12:27:17.505 INFO:teuthology.orchestra.run.smithi087.stdout:(11/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 1.3 MB/s | 364 kB 00:00 2023-01-25T12:27:17.556 INFO:teuthology.orchestra.run.smithi087.stdout:(12/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.4 MB/s | 106 kB 00:00 2023-01-25T12:27:17.616 INFO:teuthology.orchestra.run.smithi087.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.1 MB/s | 354 kB 00:00 2023-01-25T12:27:17.648 INFO:teuthology.orchestra.run.smithi087.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.4 MB/s | 44 kB 00:00 2023-01-25T12:27:17.673 INFO:teuthology.orchestra.run.smithi087.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-25T12:27:17.698 INFO:teuthology.orchestra.run.smithi087.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2023-01-25T12:27:17.774 INFO:teuthology.orchestra.run.smithi067.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 3.9 MB/s | 4.8 MB 00:01 2023-01-25T12:27:17.840 INFO:teuthology.orchestra.run.smithi087.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 416 kB/s | 59 kB 00:00 2023-01-25T12:27:17.857 INFO:teuthology.orchestra.run.smithi067.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 292 kB/s | 24 kB 00:00 2023-01-25T12:27:17.899 INFO:teuthology.orchestra.run.smithi087.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-01-25T12:27:17.924 INFO:teuthology.orchestra.run.smithi087.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 896 kB/s | 75 kB 00:00 2023-01-25T12:27:17.949 INFO:teuthology.orchestra.run.smithi087.stdout:(20/23): python3-rados-18.0.0-1986.g44e4095d.el 488 kB/s | 372 kB 00:00 2023-01-25T12:27:18.092 INFO:teuthology.orchestra.run.smithi087.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-01-25T12:27:18.108 INFO:teuthology.orchestra.run.smithi067.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 3.0 MB/s | 767 kB 00:00 2023-01-25T12:27:18.291 INFO:teuthology.orchestra.run.smithi067.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 2.5 MB/s | 474 kB 00:00 2023-01-25T12:27:18.491 INFO:teuthology.orchestra.run.smithi087.stdout:(22/23): librados2-18.0.0-1986.g44e4095d.el8.x8 6.4 MB/s | 3.6 MB 00:00 2023-01-25T12:27:18.608 INFO:teuthology.orchestra.run.smithi067.stdout:(5/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8. 6.2 MB/s | 13 MB 00:02 2023-01-25T12:27:18.783 INFO:teuthology.orchestra.run.smithi067.stdout:(6/23): python3-ceph-argparse-18.0.0-1986.g44e4 255 kB/s | 45 kB 00:00 2023-01-25T12:27:18.842 INFO:teuthology.orchestra.run.smithi067.stdout:(7/23): python3-ceph-common-18.0.0-1986.g44e409 1.9 MB/s | 115 kB 00:00 2023-01-25T12:27:18.850 INFO:teuthology.orchestra.run.smithi087.stdout:(23/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 4.0 MB/s | 3.6 MB 00:00 2023-01-25T12:27:18.853 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:27:18.854 INFO:teuthology.orchestra.run.smithi087.stdout:Total 16 MB/s | 58 MB 00:03 2023-01-25T12:27:18.891 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:27:18.909 INFO:teuthology.orchestra.run.smithi067.stdout:(8/23): python3-cephfs-18.0.0-1986.g44e4095d.el 2.8 MB/s | 190 kB 00:00 2023-01-25T12:27:18.934 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:27:18.934 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:27:19.117 INFO:teuthology.orchestra.run.smithi067.stdout:(9/23): ceph-common-18.0.0-1986.g44e4095d.el8.x 8.3 MB/s | 21 MB 00:02 2023-01-25T12:27:19.143 INFO:teuthology.orchestra.run.smithi067.stdout:(10/23): python3-rados-18.0.0-1986.g44e4095d.el 1.6 MB/s | 372 kB 00:00 2023-01-25T12:27:19.201 INFO:teuthology.orchestra.run.smithi067.stdout:(11/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.8 MB/s | 106 kB 00:00 2023-01-25T12:27:19.226 INFO:teuthology.orchestra.run.smithi067.stdout:(12/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 3.3 MB/s | 364 kB 00:00 2023-01-25T12:27:19.352 INFO:teuthology.orchestra.run.smithi067.stdout:(13/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_ 6.1 MB/s | 6.5 MB 00:01 2023-01-25T12:27:19.394 INFO:teuthology.orchestra.run.smithi067.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2023-01-25T12:27:19.419 INFO:teuthology.orchestra.run.smithi067.stdout:(15/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2023-01-25T12:27:19.444 INFO:teuthology.orchestra.run.smithi067.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 970 kB/s | 47 kB 00:00 2023-01-25T12:27:19.469 INFO:teuthology.orchestra.run.smithi067.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 387 kB/s | 44 kB 00:00 2023-01-25T12:27:19.570 INFO:teuthology.orchestra.run.smithi067.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 470 kB/s | 59 kB 00:00 2023-01-25T12:27:19.579 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:27:19.583 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:27:19.603 INFO:teuthology.orchestra.run.smithi067.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-01-25T12:27:19.654 INFO:teuthology.orchestra.run.smithi067.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-01-25T12:27:19.829 INFO:teuthology.orchestra.run.smithi067.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 6.7 MB/s | 1.7 MB 00:00 2023-01-25T12:27:19.963 INFO:teuthology.orchestra.run.smithi067.stdout:(22/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 12 MB/s | 3.6 MB 00:00 2023-01-25T12:27:20.030 INFO:teuthology.orchestra.run.smithi067.stdout:(23/23): librados2-18.0.0-1986.g44e4095d.el8.x8 8.5 MB/s | 3.6 MB 00:00 2023-01-25T12:27:20.033 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:27:20.033 INFO:teuthology.orchestra.run.smithi067.stdout:Total 17 MB/s | 58 MB 00:03 2023-01-25T12:27:20.073 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:27:20.121 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:27:20.121 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:27:20.755 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:27:20.759 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:27:21.034 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:27:21.257 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T12:27:21.494 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T12:27:21.650 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T12:27:22.043 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:27:22.061 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T12:27:22.245 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T12:27:22.267 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:22.359 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T12:27:22.631 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T12:27:22.718 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T12:27:22.792 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:27:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T12:27:23.191 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:27:23.326 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:23.350 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:23.413 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:23.581 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T12:27:23.719 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T12:27:23.750 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:27:23.816 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:27:23.952 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:27:23.952 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T12:27:24.144 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:27:24.342 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:27:24.393 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:27:24.475 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:27:24.530 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:27:24.584 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:24.653 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:24.733 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T12:27:24.818 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T12:27:24.968 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:27:25.039 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:27:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T12:27:25.414 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:27:25.535 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T12:27:25.646 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:27:25.700 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T12:27:25.777 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T12:27:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:27:25.886 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:27:26.008 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T12:27:26.084 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T12:27:26.183 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T12:27:26.375 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T12:27:26.959 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T12:27:27.184 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T12:27:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T12:27:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T12:27:27.696 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T12:27:27.926 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T12:27:29.843 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:30.075 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:30.640 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:30.969 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:27:31.101 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:27:31.399 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:31.909 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:32.510 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:27:32.844 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:27:32.933 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:27:33.125 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:50.523 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2023-01-25T12:27:50.523 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2023-01-25T12:27:50.524 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /dev 2023-01-25T12:27:50.524 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /run 2023-01-25T12:27:50.524 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2023-01-25T12:27:50.524 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2023-01-25T12:27:50.524 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2023-01-25T12:27:50.525 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2023-01-25T12:27:50.525 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /dev 2023-01-25T12:27:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:51.958 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:27:51.978 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:27:51.979 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:27:52.191 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:27:52.191 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:52.927 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:52.927 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T12:27:52.927 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T12:27:52.927 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T12:27:52.928 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:52.928 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T12:27:52.928 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T12:27:52.928 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:52.928 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T12:27:52.929 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T12:27:52.929 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T12:27:52.929 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:27:52.929 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:27:52.929 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T12:27:52.930 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T12:27:52.930 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T12:27:52.930 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T12:27:52.930 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T12:27:52.931 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T12:27:52.931 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T12:27:52.931 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T12:27:52.931 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T12:27:52.931 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:52.932 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T12:27:52.932 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T12:27:53.312 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:53.313 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2023-01-25T12:27:53.313 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2023-01-25T12:27:53.313 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-01-25T12:27:53.313 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /run 2023-01-25T12:27:53.314 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2023-01-25T12:27:53.314 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2023-01-25T12:27:53.314 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2023-01-25T12:27:53.314 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2023-01-25T12:27:53.314 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-01-25T12:27:53.315 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:53.802 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:53.802 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:53.803 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2023-01-25T12:27:53.803 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.803 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.804 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:27:53.804 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.804 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.805 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.805 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.805 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T12:27:53.806 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.806 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T12:27:53.806 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T12:27:53.807 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.807 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T12:27:53.807 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.808 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T12:27:53.808 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T12:27:53.808 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.808 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.809 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.809 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T12:27:53.809 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.809 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.809 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:53.810 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T12:27:53.810 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:53.810 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:27:54.144 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-test 2023-01-25T12:27:54.824 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:27:54.846 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:27:54.847 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:27:54.867 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:00:43 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:27:55.083 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:27:55.084 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:55.828 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:55.828 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T12:27:55.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T12:27:55.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T12:27:55.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:27:55.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T12:27:55.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T12:27:55.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:27:55.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T12:27:55.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T12:27:55.832 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T12:27:55.832 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:27:55.833 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:27:55.833 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T12:27:55.833 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T12:27:55.833 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T12:27:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T12:27:55.834 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T12:27:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T12:27:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T12:27:55.835 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T12:27:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T12:27:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:27:55.836 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T12:27:55.837 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T12:27:56.013 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:27:56.014 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:56.014 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:27:56.015 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:56.015 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:27:56.015 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T12:27:56.016 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:27:56.016 INFO:teuthology.orchestra.run.smithi087.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T12:27:56.016 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T12:27:56.017 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T12:27:56.017 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T12:27:56.017 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T12:27:56.018 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:56.018 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:27:56.018 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:27:56.018 INFO:teuthology.orchestra.run.smithi087.stdout:Install 6 Packages 2023-01-25T12:27:56.019 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:27:56.020 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 46 M 2023-01-25T12:27:56.020 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 204 M 2023-01-25T12:27:56.020 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:27:56.086 INFO:teuthology.orchestra.run.smithi087.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-01-25T12:27:56.119 INFO:teuthology.orchestra.run.smithi087.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 5.7 MB/s | 187 kB 00:00 2023-01-25T12:27:56.144 INFO:teuthology.orchestra.run.smithi087.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2023-01-25T12:27:56.170 INFO:teuthology.orchestra.run.smithi087.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-01-25T12:27:56.236 INFO:teuthology.orchestra.run.smithi087.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 624 kB/s | 137 kB 00:00 2023-01-25T12:27:56.678 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:27:56.678 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:56.679 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2023-01-25T12:27:56.679 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.679 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.679 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:27:56.680 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.680 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.680 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.680 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.680 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T12:27:56.681 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.681 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T12:27:56.681 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T12:27:56.681 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.681 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T12:27:56.682 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.682 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T12:27:56.682 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T12:27:56.682 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.682 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.683 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.683 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T12:27:56.683 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.683 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.683 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:27:56.684 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T12:27:56.684 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:56.684 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:27:57.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-test 2023-01-25T12:27:57.777 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:45 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:27:58.237 INFO:teuthology.orchestra.run.smithi087.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 20 MB/s | 45 MB 00:02 2023-01-25T12:27:58.239 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:27:58.240 INFO:teuthology.orchestra.run.smithi087.stdout:Total 21 MB/s | 46 MB 00:02 2023-01-25T12:27:58.283 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:27:58.304 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:27:58.304 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:27:58.583 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:27:58.586 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:27:58.997 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:27:58.997 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:58.998 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:27:58.998 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:58.998 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:27:58.998 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T12:27:58.998 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:27:58.999 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T12:27:58.999 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T12:27:58.999 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T12:27:58.999 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T12:27:58.999 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T12:27:59.000 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:59.000 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:27:59.000 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:27:59.000 INFO:teuthology.orchestra.run.smithi067.stdout:Install 6 Packages 2023-01-25T12:27:59.000 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:27:59.001 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 46 M 2023-01-25T12:27:59.001 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 204 M 2023-01-25T12:27:59.001 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:27:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:27:59.085 INFO:teuthology.orchestra.run.smithi067.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.3 MB/s | 202 kB 00:00 2023-01-25T12:27:59.111 INFO:teuthology.orchestra.run.smithi067.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.6 MB/s | 187 kB 00:00 2023-01-25T12:27:59.144 INFO:teuthology.orchestra.run.smithi067.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.5 MB/s | 323 kB 00:00 2023-01-25T12:27:59.170 INFO:teuthology.orchestra.run.smithi067.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-01-25T12:27:59.228 INFO:teuthology.orchestra.run.smithi067.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 601 kB/s | 137 kB 00:00 2023-01-25T12:27:59.306 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T12:27:59.498 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T12:27:59.687 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T12:27:59.765 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T12:27:59.974 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T12:28:00.128 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:28:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 15 MB/s | 45 MB 00:03 2023-01-25T12:28:02.031 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:02.031 INFO:teuthology.orchestra.run.smithi067.stdout:Total 15 MB/s | 46 MB 00:03 2023-01-25T12:28:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:02.098 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:02.405 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:02.409 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:02.920 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:03.127 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T12:28:03.326 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T12:28:03.480 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T12:28:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T12:28:03.767 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T12:28:03.946 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:28:04.721 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:28:04.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:28:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:28:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T12:28:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T12:28:05.263 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T12:28:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T12:28:05.264 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T12:28:05.804 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T12:28:05.804 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:05.804 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:05.805 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:05.805 INFO:teuthology.orchestra.run.smithi087.stdout: jq-1.6-6.el8.x86_64 2023-01-25T12:28:05.805 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:05.805 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T12:28:05.806 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T12:28:05.806 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T12:28:05.806 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:05.806 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:06.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph 2023-01-25T12:28:06.749 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:07.890 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:07.894 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2023-01-25T12:28:07.894 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:28:07.894 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2023-01-25T12:28:07.894 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:07.894 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T12:28:07.895 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:28:07.895 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T12:28:07.895 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T12:28:07.895 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T12:28:07.895 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T12:28:07.896 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T12:28:07.896 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T12:28:07.896 INFO:teuthology.orchestra.run.smithi087.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T12:28:07.896 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T12:28:07.897 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T12:28:07.897 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T12:28:07.897 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T12:28:07.897 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T12:28:07.897 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T12:28:07.898 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T12:28:07.898 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T12:28:07.898 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T12:28:07.898 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T12:28:07.898 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T12:28:07.899 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T12:28:07.899 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T12:28:07.899 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T12:28:07.899 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T12:28:07.899 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T12:28:07.900 INFO:teuthology.orchestra.run.smithi087.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T12:28:07.900 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T12:28:07.900 INFO:teuthology.orchestra.run.smithi087.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T12:28:07.900 INFO:teuthology.orchestra.run.smithi087.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T12:28:07.900 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T12:28:07.901 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T12:28:07.901 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T12:28:07.901 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T12:28:07.901 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:07.902 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:07.902 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2023-01-25T12:28:07.902 INFO:teuthology.orchestra.run.smithi087.stdout:Install 32 Packages 2023-01-25T12:28:07.902 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:07.903 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 28 M 2023-01-25T12:28:07.903 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 107 M 2023-01-25T12:28:07.903 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:08.029 INFO:teuthology.orchestra.run.smithi087.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 50 kB/s | 6.4 kB 00:00 2023-01-25T12:28:08.288 INFO:teuthology.orchestra.run.smithi087.stdout:(2/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 4.0 MB/s | 1.5 MB 00:00 2023-01-25T12:28:08.388 INFO:teuthology.orchestra.run.smithi087.stdout:(3/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 4.7 MB/s | 2.3 MB 00:00 2023-01-25T12:28:08.455 INFO:teuthology.orchestra.run.smithi087.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 3.5 MB/s | 236 kB 00:00 2023-01-25T12:28:08.513 INFO:teuthology.orchestra.run.smithi087.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T12:28:08.538 INFO:teuthology.orchestra.run.smithi087.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T12:28:08.564 INFO:teuthology.orchestra.run.smithi087.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-01-25T12:28:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:28:08.597 INFO:teuthology.orchestra.run.smithi087.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-01-25T12:28:08.639 INFO:teuthology.orchestra.run.smithi087.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 11 MB/s | 457 kB 00:00 2023-01-25T12:28:08.673 INFO:teuthology.orchestra.run.smithi087.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 7.0 MB/s | 237 kB 00:00 2023-01-25T12:28:08.707 INFO:teuthology.orchestra.run.smithi087.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T12:28:08.732 INFO:teuthology.orchestra.run.smithi087.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T12:28:08.824 INFO:teuthology.orchestra.run.smithi087.stdout:(13/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86 4.9 MB/s | 3.9 MB 00:00 2023-01-25T12:28:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:28:08.933 INFO:teuthology.orchestra.run.smithi087.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 217 kB/s | 44 kB 00:00 2023-01-25T12:28:08.991 INFO:teuthology.orchestra.run.smithi087.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.1 MB/s | 185 kB 00:00 2023-01-25T12:28:09.033 INFO:teuthology.orchestra.run.smithi087.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2023-01-25T12:28:09.083 INFO:teuthology.orchestra.run.smithi087.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.1 MB/s | 384 kB 00:00 2023-01-25T12:28:09.134 INFO:teuthology.orchestra.run.smithi087.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-01-25T12:28:09.159 INFO:teuthology.orchestra.run.smithi087.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 233 kB/s | 18 kB 00:00 2023-01-25T12:28:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:28:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T12:28:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T12:28:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T12:28:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T12:28:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T12:28:09.194 INFO:teuthology.orchestra.run.smithi087.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 830 kB/s | 49 kB 00:00 2023-01-25T12:28:09.218 INFO:teuthology.orchestra.run.smithi087.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 999 kB/s | 59 kB 00:00 2023-01-25T12:28:09.277 INFO:teuthology.orchestra.run.smithi087.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-01-25T12:28:09.310 INFO:teuthology.orchestra.run.smithi087.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-01-25T12:28:09.335 INFO:teuthology.orchestra.run.smithi087.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 319 kB/s | 19 kB 00:00 2023-01-25T12:28:09.502 INFO:teuthology.orchestra.run.smithi087.stdout:(25/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 14 MB/s | 17 MB 00:01 2023-01-25T12:28:09.528 INFO:teuthology.orchestra.run.smithi087.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 110 kB/s | 24 kB 00:00 2023-01-25T12:28:09.553 INFO:teuthology.orchestra.run.smithi087.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 134 kB/s | 29 kB 00:00 2023-01-25T12:28:09.628 INFO:teuthology.orchestra.run.smithi087.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 220 kB/s | 27 kB 00:00 2023-01-25T12:28:09.653 INFO:teuthology.orchestra.run.smithi087.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.4 MB/s | 241 kB 00:00 2023-01-25T12:28:09.679 INFO:teuthology.orchestra.run.smithi087.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.0 MB/s | 251 kB 00:00 2023-01-25T12:28:09.704 INFO:teuthology.orchestra.run.smithi087.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2023-01-25T12:28:09.729 INFO:teuthology.orchestra.run.smithi087.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2023-01-25T12:28:09.731 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:09.732 INFO:teuthology.orchestra.run.smithi087.stdout:Total 15 MB/s | 28 MB 00:01 2023-01-25T12:28:09.775 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T12:28:09.776 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:09.776 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:09.776 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:09.776 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-6.el8.x86_64 2023-01-25T12:28:09.777 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:09.777 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T12:28:09.777 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T12:28:09.777 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T12:28:09.777 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:09.778 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:09.807 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:09.844 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:09.844 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:10.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph 2023-01-25T12:28:10.226 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:10.229 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:11.421 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:11.691 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T12:28:11.943 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T12:28:12.016 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:12.019 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2023-01-25T12:28:12.019 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:28:12.019 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2023-01-25T12:28:12.019 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:28:12.020 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T12:28:12.020 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:28:12.020 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T12:28:12.020 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T12:28:12.020 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T12:28:12.021 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T12:28:12.021 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T12:28:12.021 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T12:28:12.021 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T12:28:12.021 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T12:28:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T12:28:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T12:28:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T12:28:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T12:28:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T12:28:12.023 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T12:28:12.023 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T12:28:12.023 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T12:28:12.023 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T12:28:12.023 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T12:28:12.024 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T12:28:12.024 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T12:28:12.024 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T12:28:12.024 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T12:28:12.024 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T12:28:12.025 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T12:28:12.025 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T12:28:12.025 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T12:28:12.025 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T12:28:12.025 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T12:28:12.026 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T12:28:12.026 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T12:28:12.026 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T12:28:12.026 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:12.026 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:28:12.027 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2023-01-25T12:28:12.027 INFO:teuthology.orchestra.run.smithi067.stdout:Install 32 Packages 2023-01-25T12:28:12.027 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:12.028 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 28 M 2023-01-25T12:28:12.028 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 107 M 2023-01-25T12:28:12.028 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:28:12.155 INFO:teuthology.orchestra.run.smithi067.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2023-01-25T12:28:12.190 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T12:28:12.397 INFO:teuthology.orchestra.run.smithi067.stdout:(2/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 6.1 MB/s | 2.3 MB 00:00 2023-01-25T12:28:12.431 INFO:teuthology.orchestra.run.smithi067.stdout:(3/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 3.8 MB/s | 1.5 MB 00:00 2023-01-25T12:28:12.467 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T12:28:12.506 INFO:teuthology.orchestra.run.smithi067.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 3.1 MB/s | 236 kB 00:00 2023-01-25T12:28:12.565 INFO:teuthology.orchestra.run.smithi067.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T12:28:12.581 INFO:teuthology.orchestra.run.smithi067.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 2.3 MB/s | 39 kB 00:00 2023-01-25T12:28:12.631 INFO:teuthology.orchestra.run.smithi067.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.0 MB/s | 103 kB 00:00 2023-01-25T12:28:12.657 INFO:teuthology.orchestra.run.smithi067.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-25T12:28:12.672 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T12:28:12.808 INFO:teuthology.orchestra.run.smithi067.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 3.0 MB/s | 457 kB 00:00 2023-01-25T12:28:12.833 INFO:teuthology.orchestra.run.smithi067.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.4 MB/s | 237 kB 00:00 2023-01-25T12:28:12.836 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T12:28:12.866 INFO:teuthology.orchestra.run.smithi067.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T12:28:12.892 INFO:teuthology.orchestra.run.smithi067.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T12:28:13.013 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T12:28:13.025 INFO:teuthology.orchestra.run.smithi067.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 327 kB/s | 44 kB 00:00 2023-01-25T12:28:13.134 INFO:teuthology.orchestra.run.smithi067.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.7 MB/s | 185 kB 00:00 2023-01-25T12:28:13.182 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T12:28:13.193 INFO:teuthology.orchestra.run.smithi067.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.9 MB/s | 173 kB 00:00 2023-01-25T12:28:13.277 INFO:teuthology.orchestra.run.smithi067.stdout:(16/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86 3.5 MB/s | 3.9 MB 00:01 2023-01-25T12:28:13.382 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T12:28:13.610 INFO:teuthology.orchestra.run.smithi067.stdout:(17/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 14 MB/s | 17 MB 00:01 2023-01-25T12:28:13.636 INFO:teuthology.orchestra.run.smithi067.stdout:(18/32): python3-cherrypy-18.4.0-1.el8.noarch.r 869 kB/s | 384 kB 00:00 2023-01-25T12:28:13.661 INFO:teuthology.orchestra.run.smithi067.stdout:(19/32): python3-jaraco-6.2-6.el8.noarch.rpm 29 kB/s | 11 kB 00:00 2023-01-25T12:28:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T12:28:13.686 INFO:teuthology.orchestra.run.smithi067.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 970 kB/s | 49 kB 00:00 2023-01-25T12:28:13.736 INFO:teuthology.orchestra.run.smithi067.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 780 kB/s | 59 kB 00:00 2023-01-25T12:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout:(22/32): python3-jaraco-functools-2.0-4.el8.noa 116 kB/s | 18 kB 00:00 2023-01-25T12:28:13.787 INFO:teuthology.orchestra.run.smithi067.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-01-25T12:28:13.813 INFO:teuthology.orchestra.run.smithi067.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2023-01-25T12:28:13.838 INFO:teuthology.orchestra.run.smithi067.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 248 kB/s | 19 kB 00:00 2023-01-25T12:28:13.863 INFO:teuthology.orchestra.run.smithi067.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 316 kB/s | 24 kB 00:00 2023-01-25T12:28:13.905 INFO:teuthology.orchestra.run.smithi067.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 315 kB/s | 29 kB 00:00 2023-01-25T12:28:13.917 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T12:28:13.938 INFO:teuthology.orchestra.run.smithi067.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 273 kB/s | 27 kB 00:00 2023-01-25T12:28:13.963 INFO:teuthology.orchestra.run.smithi067.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.3 MB/s | 241 kB 00:00 2023-01-25T12:28:14.014 INFO:teuthology.orchestra.run.smithi067.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 458 kB/s | 23 kB 00:00 2023-01-25T12:28:14.039 INFO:teuthology.orchestra.run.smithi067.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 851 kB/s | 85 kB 00:00 2023-01-25T12:28:14.064 INFO:teuthology.orchestra.run.smithi067.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2023-01-25T12:28:14.067 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:14.067 INFO:teuthology.orchestra.run.smithi067.stdout:Total 14 MB/s | 28 MB 00:02 2023-01-25T12:28:14.152 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T12:28:14.154 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:14.195 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:14.195 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:14.554 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T12:28:14.596 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:14.600 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:14.788 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T12:28:14.968 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T12:28:15.234 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T12:28:15.461 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T12:28:15.734 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T12:28:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:15.974 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T12:28:16.170 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T12:28:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T12:28:16.390 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T12:28:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T12:28:16.616 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T12:28:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T12:28:16.830 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T12:28:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T12:28:17.053 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T12:28:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T12:28:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T12:28:17.414 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T12:28:17.581 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T12:28:17.749 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T12:28:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T12:28:18.003 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T12:28:18.115 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T12:28:18.307 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T12:28:18.311 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:28:18.554 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T12:28:18.808 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T12:28:19.246 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T12:28:19.516 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T12:28:19.703 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T12:28:19.765 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:28:19.979 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:28:19.989 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T12:28:20.230 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T12:28:20.374 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:28:20.543 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:28:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T12:28:20.759 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:28:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T12:28:20.934 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:28:20.953 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:28:21.039 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T12:28:21.074 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:28:21.283 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T12:28:21.519 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:28:21.520 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T12:28:21.520 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T12:28:21.520 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T12:28:21.520 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T12:28:21.521 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T12:28:21.521 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T12:28:21.521 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T12:28:21.521 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T12:28:21.521 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T12:28:21.522 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T12:28:21.522 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T12:28:21.522 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T12:28:21.522 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T12:28:21.523 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T12:28:21.523 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T12:28:21.523 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T12:28:21.523 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T12:28:21.523 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T12:28:21.524 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T12:28:21.524 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T12:28:21.524 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T12:28:21.524 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T12:28:21.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T12:28:21.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T12:28:21.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T12:28:21.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T12:28:21.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T12:28:21.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T12:28:21.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T12:28:21.527 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T12:28:21.527 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T12:28:21.533 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T12:28:21.783 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T12:28:22.010 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T12:28:22.414 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T12:28:22.517 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T12:28:22.517 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:22.518 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:22.518 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:22.518 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:22.518 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:22.518 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:22.519 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:22.519 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:22.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T12:28:22.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T12:28:22.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T12:28:22.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T12:28:22.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T12:28:22.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T12:28:22.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T12:28:22.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T12:28:22.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T12:28:22.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T12:28:22.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T12:28:22.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T12:28:22.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T12:28:22.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T12:28:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T12:28:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T12:28:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T12:28:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T12:28:22.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T12:28:22.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T12:28:22.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T12:28:22.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T12:28:22.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T12:28:22.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T12:28:22.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T12:28:22.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T12:28:22.525 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:22.525 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:22.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-base 2023-01-25T12:28:22.789 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T12:28:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T12:28:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:28:23.468 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:12 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:24.502 INFO:teuthology.orchestra.run.smithi087.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:28:24.601 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:24.602 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:28:24.602 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:24.695 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install cephadm 2023-01-25T12:28:24.996 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:28:25.231 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:28:25.412 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:14 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:25.655 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:28:25.836 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:28:26.063 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:28:26.236 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:28:26.255 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:28:26.385 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:28:26.544 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:26.545 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:26.545 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:28:26.545 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:26.545 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:26.545 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T12:28:26.546 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:26.546 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:26.546 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:26.546 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:28:26.547 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:26.547 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 82 k 2023-01-25T12:28:26.547 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 374 k 2023-01-25T12:28:26.547 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:26.715 INFO:teuthology.orchestra.run.smithi087.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 484 kB/s | 82 kB 00:00 2023-01-25T12:28:26.716 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:26.716 INFO:teuthology.orchestra.run.smithi087.stdout:Total 481 kB/s | 82 kB 00:00 2023-01-25T12:28:26.717 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:26.724 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:26.724 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:26.726 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:26.729 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:26.797 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:28:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T12:28:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T12:28:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T12:28:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T12:28:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T12:28:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T12:28:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T12:28:26.827 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T12:28:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T12:28:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T12:28:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T12:28:26.828 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T12:28:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T12:28:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T12:28:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T12:28:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T12:28:26.829 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T12:28:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T12:28:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T12:28:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T12:28:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T12:28:26.830 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T12:28:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T12:28:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T12:28:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T12:28:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T12:28:27.763 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T12:28:27.763 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:27.763 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:27.764 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:27.764 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:27.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:27.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:27.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:27.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:27.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T12:28:27.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T12:28:27.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T12:28:27.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T12:28:27.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T12:28:27.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T12:28:27.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T12:28:27.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T12:28:27.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T12:28:27.768 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T12:28:27.768 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T12:28:27.768 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T12:28:27.768 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T12:28:27.768 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T12:28:27.769 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T12:28:27.769 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T12:28:27.769 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T12:28:27.769 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T12:28:27.769 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T12:28:27.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T12:28:27.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T12:28:27.771 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:27.771 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:27.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-base 2023-01-25T12:28:28.287 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:28.444 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:28.681 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:28.728 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:16 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:29.116 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:29.117 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:29.117 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:29.117 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:29.117 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:29.117 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:29.380 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T12:28:29.839 INFO:teuthology.orchestra.run.smithi067.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:28:29.953 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:29.954 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:28:29.955 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:30.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install cephadm 2023-01-25T12:28:30.107 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:19 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:30.789 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:18 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:31.252 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:31.253 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:31.253 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repo Size 2023-01-25T12:28:31.254 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:31.254 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:31.254 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T12:28:31.255 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:31.255 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:31.255 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:31.256 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:28:31.257 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:31.257 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 165 k 2023-01-25T12:28:31.257 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 503 k 2023-01-25T12:28:31.258 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:31.620 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 452 kB/s | 165 kB 00:00 2023-01-25T12:28:31.621 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:31.621 INFO:teuthology.orchestra.run.smithi087.stdout:Total 450 kB/s | 165 kB 00:00 2023-01-25T12:28:31.621 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:31.632 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:31.633 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:31.681 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:31.685 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:31.815 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:31.991 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:32.006 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:32.007 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:32.007 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:28:32.007 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:32.008 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:28:32.008 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T12:28:32.008 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:32.008 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:28:32.008 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:32.009 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:28:32.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:32.009 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 82 k 2023-01-25T12:28:32.009 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 374 k 2023-01-25T12:28:32.009 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:28:32.171 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 508 kB/s | 82 kB 00:00 2023-01-25T12:28:32.171 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:32.171 INFO:teuthology.orchestra.run.smithi067.stdout:Total 504 kB/s | 82 kB 00:00 2023-01-25T12:28:32.172 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:32.179 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:32.180 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:32.182 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:32.185 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:32.245 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:32.425 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:32.884 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:32.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:32.884 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:32.885 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:32.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:32.885 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:33.065 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-mgr 2023-01-25T12:28:33.795 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:22 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:33.935 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:34.146 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:34.398 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:34.838 INFO:teuthology.orchestra.run.smithi087.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:28:34.928 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:28:34.929 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:34.929 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:34.929 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:34.930 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:34.930 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:34.937 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:34.938 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:28:34.939 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:35.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T12:28:35.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T12:28:35.740 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:24 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:35.922 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:23 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:36.875 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:36.876 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:36.876 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:28:36.876 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:36.877 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:36.877 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T12:28:36.877 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:28:36.877 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T12:28:36.877 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T12:28:36.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T12:28:36.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T12:28:36.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T12:28:36.878 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:36.878 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:36.879 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:28:36.879 INFO:teuthology.orchestra.run.smithi087.stdout:Install 6 Packages 2023-01-25T12:28:36.879 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:36.881 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 2.0 M 2023-01-25T12:28:36.881 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 8.2 M 2023-01-25T12:28:36.881 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:37.145 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:37.145 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:37.146 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2023-01-25T12:28:37.146 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:37.146 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:28:37.146 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T12:28:37.147 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:37.147 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:28:37.147 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:37.147 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:28:37.147 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:37.148 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 165 k 2023-01-25T12:28:37.148 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 503 k 2023-01-25T12:28:37.148 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:28:37.360 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 776 kB/s | 165 kB 00:00 2023-01-25T12:28:37.361 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:37.361 INFO:teuthology.orchestra.run.smithi067.stdout:Total 772 kB/s | 165 kB 00:00 2023-01-25T12:28:37.361 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:37.373 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:37.373 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:37.426 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:37.429 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:37.576 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:37.617 INFO:teuthology.orchestra.run.smithi087.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 31 kB/s | 23 kB 00:00 2023-01-25T12:28:37.700 INFO:teuthology.orchestra.run.smithi087.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 18 kB/s | 15 kB 00:00 2023-01-25T12:28:37.743 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:37.751 INFO:teuthology.orchestra.run.smithi087.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2023-01-25T12:28:37.851 INFO:teuthology.orchestra.run.smithi087.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 224 kB/s | 34 kB 00:00 2023-01-25T12:28:37.968 INFO:teuthology.orchestra.run.smithi087.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 907 kB/s | 196 kB 00:00 2023-01-25T12:28:38.168 INFO:teuthology.orchestra.run.smithi087.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 1.3 MB/s | 1.7 MB 00:01 2023-01-25T12:28:38.171 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:38.171 INFO:teuthology.orchestra.run.smithi087.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2023-01-25T12:28:38.183 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:38.192 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:38.193 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:38.279 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:38.283 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:38.570 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:38.694 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:28:38.695 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:38.695 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:38.695 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:28:38.695 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:38.696 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:38.788 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T12:28:38.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr 2023-01-25T12:28:38.991 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T12:28:39.151 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T12:28:39.263 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T12:28:39.621 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T12:28:39.647 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:27 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:39.817 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:28:40.117 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:28:40.118 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T12:28:40.118 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T12:28:40.118 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T12:28:40.118 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T12:28:40.119 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T12:28:40.586 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T12:28:40.587 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:40.587 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:40.587 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:40.587 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:40.587 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:40.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T12:28:40.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T12:28:40.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T12:28:40.588 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:40.588 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:40.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T12:28:40.777 INFO:teuthology.orchestra.run.smithi067.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:28:40.888 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:40.890 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:28:40.890 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:40.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T12:28:41.417 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:41.744 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:29 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:42.557 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:42.558 INFO:teuthology.orchestra.run.smithi087.stdout:===================================================================================================== 2023-01-25T12:28:42.558 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:28:42.558 INFO:teuthology.orchestra.run.smithi087.stdout:===================================================================================================== 2023-01-25T12:28:42.559 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:42.559 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T12:28:42.559 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:28:42.559 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T12:28:42.559 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T12:28:42.560 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T12:28:42.560 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T12:28:42.560 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T12:28:42.560 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T12:28:42.560 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T12:28:42.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:42.561 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:42.561 INFO:teuthology.orchestra.run.smithi087.stdout:===================================================================================================== 2023-01-25T12:28:42.561 INFO:teuthology.orchestra.run.smithi087.stdout:Install 8 Packages 2023-01-25T12:28:42.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:42.562 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 35 M 2023-01-25T12:28:42.562 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 210 M 2023-01-25T12:28:42.564 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:42.780 INFO:teuthology.orchestra.run.smithi087.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-01-25T12:28:42.886 INFO:teuthology.orchestra.run.smithi087.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 35 MB/s | 3.7 MB 00:00 2023-01-25T12:28:42.920 INFO:teuthology.orchestra.run.smithi087.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.7 MB/s | 225 kB 00:00 2023-01-25T12:28:42.972 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:42.973 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:42.973 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:28:42.973 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:42.974 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:28:42.974 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T12:28:42.975 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:28:42.975 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T12:28:42.975 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T12:28:42.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T12:28:42.975 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T12:28:42.976 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T12:28:42.976 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:42.976 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:28:42.976 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:28:42.977 INFO:teuthology.orchestra.run.smithi067.stdout:Install 6 Packages 2023-01-25T12:28:42.977 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:42.977 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 2.0 M 2023-01-25T12:28:42.978 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 8.2 M 2023-01-25T12:28:42.978 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:28:43.029 INFO:teuthology.orchestra.run.smithi087.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.9 MB 00:00 2023-01-25T12:28:43.087 INFO:teuthology.orchestra.run.smithi087.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 11 MB/s | 645 kB 00:00 2023-01-25T12:28:43.129 INFO:teuthology.orchestra.run.smithi087.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 4.0 MB/s | 172 kB 00:00 2023-01-25T12:28:43.206 INFO:teuthology.orchestra.run.smithi067.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 100 kB/s | 23 kB 00:00 2023-01-25T12:28:43.231 INFO:teuthology.orchestra.run.smithi067.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 57 kB/s | 15 kB 00:00 2023-01-25T12:28:43.246 INFO:teuthology.orchestra.run.smithi087.stdout:(7/8): ceph-mgr-diskprediction-local-18.0.0-198 11 MB/s | 7.4 MB 00:00 2023-01-25T12:28:43.306 INFO:teuthology.orchestra.run.smithi067.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 426 kB/s | 43 kB 00:00 2023-01-25T12:28:43.340 INFO:teuthology.orchestra.run.smithi067.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 310 kB/s | 34 kB 00:00 2023-01-25T12:28:43.382 INFO:teuthology.orchestra.run.smithi067.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 4.2 MB/s | 1.7 MB 00:00 2023-01-25T12:28:43.421 INFO:teuthology.orchestra.run.smithi087.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 27 MB/s | 14 MB 00:00 2023-01-25T12:28:43.424 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:43.424 INFO:teuthology.orchestra.run.smithi087.stdout:Total 41 MB/s | 35 MB 00:00 2023-01-25T12:28:43.507 INFO:teuthology.orchestra.run.smithi067.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 981 kB/s | 196 kB 00:00 2023-01-25T12:28:43.510 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:43.511 INFO:teuthology.orchestra.run.smithi067.stdout:Total 3.8 MB/s | 2.0 MB 00:00 2023-01-25T12:28:43.520 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:43.529 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:43.529 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:43.617 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:43.620 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:43.655 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:43.673 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:43.673 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:43.910 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:44.070 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:44.074 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T12:28:44.377 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T12:28:44.594 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T12:28:44.674 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:44.715 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T12:28:44.806 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T12:28:44.935 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T12:28:45.064 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T12:28:45.123 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T12:28:45.363 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:28:45.665 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T12:28:45.686 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:28:45.686 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T12:28:45.686 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T12:28:45.686 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T12:28:45.687 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T12:28:45.687 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T12:28:45.839 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:46.216 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T12:28:46.217 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:46.217 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:46.217 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:46.217 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:46.218 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:46.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T12:28:46.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T12:28:46.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T12:28:46.218 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:46.219 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:46.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T12:28:46.445 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:46.639 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T12:28:47.089 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:35 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:47.326 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T12:28:47.651 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T12:28:48.205 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:28:48.319 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:28:48.321 INFO:teuthology.orchestra.run.smithi067.stdout:===================================================================================================== 2023-01-25T12:28:48.321 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:28:48.321 INFO:teuthology.orchestra.run.smithi067.stdout:===================================================================================================== 2023-01-25T12:28:48.321 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:28:48.322 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T12:28:48.322 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:28:48.322 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T12:28:48.322 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T12:28:48.322 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T12:28:48.323 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T12:28:48.323 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T12:28:48.323 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T12:28:48.323 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T12:28:48.323 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:48.324 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:28:48.324 INFO:teuthology.orchestra.run.smithi067.stdout:===================================================================================================== 2023-01-25T12:28:48.324 INFO:teuthology.orchestra.run.smithi067.stdout:Install 8 Packages 2023-01-25T12:28:48.324 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:48.325 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 35 M 2023-01-25T12:28:48.325 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 210 M 2023-01-25T12:28:48.325 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:28:48.504 INFO:teuthology.orchestra.run.smithi067.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-01-25T12:28:48.596 INFO:teuthology.orchestra.run.smithi067.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2023-01-25T12:28:48.629 INFO:teuthology.orchestra.run.smithi067.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-01-25T12:28:48.721 INFO:teuthology.orchestra.run.smithi067.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2023-01-25T12:28:48.789 INFO:teuthology.orchestra.run.smithi067.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 9.4 MB/s | 645 kB 00:00 2023-01-25T12:28:48.822 INFO:teuthology.orchestra.run.smithi067.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2023-01-25T12:28:49.056 INFO:teuthology.orchestra.run.smithi067.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 32 MB/s | 14 MB 00:00 2023-01-25T12:28:49.540 INFO:teuthology.orchestra.run.smithi067.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-198 6.1 MB/s | 7.4 MB 00:01 2023-01-25T12:28:49.542 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:49.542 INFO:teuthology.orchestra.run.smithi067.stdout:Total 29 MB/s | 35 MB 00:01 2023-01-25T12:28:49.782 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:28:49.801 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:28:49.802 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:28:49.983 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:28:50.209 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:28:50.213 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:28:50.799 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:28:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T12:28:51.132 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T12:28:51.260 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T12:28:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T12:28:51.447 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:28:51.789 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:28:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T12:28:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T12:28:51.790 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:51.791 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T12:28:51.791 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T12:28:51.791 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T12:28:51.791 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T12:28:52.076 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T12:28:52.276 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T12:28:52.277 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:52.277 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:28:52.277 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:52.277 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T12:28:52.278 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T12:28:52.278 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T12:28:52.278 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T12:28:52.278 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T12:28:52.278 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T12:28:52.279 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T12:28:52.279 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:52.279 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:28:52.282 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:52.398 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-mgr-rook 2023-01-25T12:28:52.944 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:53.124 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T12:28:53.124 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:28:53.844 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T12:28:54.207 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T12:28:54.265 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:28:54.269 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================== 2023-01-25T12:28:54.269 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:28:54.270 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================== 2023-01-25T12:28:54.270 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:28:54.270 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T12:28:54.271 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:28:54.271 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T12:28:54.271 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T12:28:54.271 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T12:28:54.271 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T12:28:54.272 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T12:28:54.272 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T12:28:54.272 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T12:28:54.272 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T12:28:54.272 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T12:28:54.273 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T12:28:54.273 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T12:28:54.273 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T12:28:54.273 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:54.273 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:28:54.274 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================== 2023-01-25T12:28:54.274 INFO:teuthology.orchestra.run.smithi087.stdout:Install 13 Packages 2023-01-25T12:28:54.274 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:28:54.274 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 1.9 M 2023-01-25T12:28:54.275 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 25 M 2023-01-25T12:28:54.275 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:28:54.346 INFO:teuthology.orchestra.run.smithi087.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 267 kB/s | 20 kB 00:00 2023-01-25T12:28:54.371 INFO:teuthology.orchestra.run.smithi087.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 265 kB/s | 27 kB 00:00 2023-01-25T12:28:54.396 INFO:teuthology.orchestra.run.smithi087.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-25T12:28:54.422 INFO:teuthology.orchestra.run.smithi087.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T12:28:54.447 INFO:teuthology.orchestra.run.smithi087.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T12:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:28:54.464 INFO:teuthology.orchestra.run.smithi087.stdout:(6/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 245 kB/s | 47 kB 00:00 2023-01-25T12:28:54.481 INFO:teuthology.orchestra.run.smithi087.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 722 kB/s | 43 kB 00:00 2023-01-25T12:28:54.565 INFO:teuthology.orchestra.run.smithi087.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 162 kB/s | 16 kB 00:00 2023-01-25T12:28:54.598 INFO:teuthology.orchestra.run.smithi087.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 218 kB/s | 33 kB 00:00 2023-01-25T12:28:54.624 INFO:teuthology.orchestra.run.smithi087.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 571 kB/s | 81 kB 00:00 2023-01-25T12:28:54.666 INFO:teuthology.orchestra.run.smithi087.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.4 MB/s | 61 kB 00:00 2023-01-25T12:28:54.683 INFO:teuthology.orchestra.run.smithi087.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 731 kB/s | 61 kB 00:00 2023-01-25T12:28:54.958 INFO:teuthology.orchestra.run.smithi087.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.0 MB/s | 1.2 MB 00:00 2023-01-25T12:28:54.960 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:28:54.960 INFO:teuthology.orchestra.run.smithi087.stdout:Total 2.8 MB/s | 1.9 MB 00:00 2023-01-25T12:28:55.012 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:28:55.027 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:28:55.028 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:28:55.123 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:28:55.127 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:28:55.618 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:28:55.873 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T12:28:56.072 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T12:28:56.290 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T12:28:56.339 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:28:56.494 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T12:28:56.668 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T12:28:56.874 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T12:28:57.174 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T12:28:57.429 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T12:28:57.650 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T12:28:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:28:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T12:28:58.515 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:28:58.515 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T12:28:58.516 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T12:28:58.516 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T12:28:58.516 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T12:28:58.517 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T12:28:58.517 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T12:28:58.517 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T12:28:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T12:28:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T12:28:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T12:28:59.064 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T12:28:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:59.065 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:28:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:28:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T12:28:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T12:28:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T12:28:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T12:28:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T12:28:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T12:28:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T12:28:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:28:59.067 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:28:59.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-rook 2023-01-25T12:28:59.938 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:47 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:28:59.998 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:29:00.312 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:29:00.312 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T12:29:00.312 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T12:29:00.312 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T12:29:00.313 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T12:29:00.313 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T12:29:00.313 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T12:29:00.313 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T12:29:00.314 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T12:29:00.314 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T12:29:00.314 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T12:29:00.314 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T12:29:00.314 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T12:29:00.876 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T12:29:00.876 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:00.876 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:00.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:00.877 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T12:29:00.877 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T12:29:00.877 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T12:29:00.877 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T12:29:00.877 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T12:29:00.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T12:29:00.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T12:29:00.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T12:29:00.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T12:29:00.878 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T12:29:00.879 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T12:29:00.879 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T12:29:00.879 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:00.879 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:01.002 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T12:29:01.165 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:01.166 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================== 2023-01-25T12:29:01.166 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:29:01.167 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================== 2023-01-25T12:29:01.167 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:01.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T12:29:01.167 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:29:01.168 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T12:29:01.168 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T12:29:01.168 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T12:29:01.168 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T12:29:01.168 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T12:29:01.169 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T12:29:01.169 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T12:29:01.169 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T12:29:01.169 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T12:29:01.169 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T12:29:01.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T12:29:01.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T12:29:01.170 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:01.170 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:01.170 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================== 2023-01-25T12:29:01.171 INFO:teuthology.orchestra.run.smithi067.stdout:Install 13 Packages 2023-01-25T12:29:01.171 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:01.171 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 1.9 M 2023-01-25T12:29:01.172 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 25 M 2023-01-25T12:29:01.172 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:01.247 INFO:teuthology.orchestra.run.smithi067.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 347 kB/s | 27 kB 00:00 2023-01-25T12:29:01.272 INFO:teuthology.orchestra.run.smithi067.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 196 kB/s | 20 kB 00:00 2023-01-25T12:29:01.297 INFO:teuthology.orchestra.run.smithi067.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-25T12:29:01.324 INFO:teuthology.orchestra.run.smithi067.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T12:29:01.373 INFO:teuthology.orchestra.run.smithi067.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 854 kB/s | 43 kB 00:00 2023-01-25T12:29:01.398 INFO:teuthology.orchestra.run.smithi067.stdout:(6/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 207 kB/s | 47 kB 00:00 2023-01-25T12:29:01.424 INFO:teuthology.orchestra.run.smithi067.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.2 MB/s | 155 kB 00:00 2023-01-25T12:29:01.508 INFO:teuthology.orchestra.run.smithi067.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 246 kB/s | 33 kB 00:00 2023-01-25T12:29:01.541 INFO:teuthology.orchestra.run.smithi067.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 114 kB/s | 16 kB 00:00 2023-01-25T12:29:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 570 kB/s | 81 kB 00:00 2023-01-25T12:29:01.608 INFO:teuthology.orchestra.run.smithi067.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.4 MB/s | 61 kB 00:00 2023-01-25T12:29:01.633 INFO:teuthology.orchestra.run.smithi067.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 667 kB/s | 61 kB 00:00 2023-01-25T12:29:01.729 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:50 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:01.892 INFO:teuthology.orchestra.run.smithi067.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.0 MB/s | 1.2 MB 00:00 2023-01-25T12:29:01.894 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:01.894 INFO:teuthology.orchestra.run.smithi067.stdout:Total 2.7 MB/s | 1.9 MB 00:00 2023-01-25T12:29:01.953 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:01.970 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:01.971 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:02.076 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:02.080 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:02.617 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:02.865 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:02.867 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:02.868 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:29:02.868 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:02.868 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:02.868 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T12:29:02.869 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:29:02.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T12:29:02.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T12:29:02.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T12:29:02.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T12:29:02.870 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:02.870 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:02.870 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:02.870 INFO:teuthology.orchestra.run.smithi087.stdout:Install 5 Packages 2023-01-25T12:29:02.870 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:02.871 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 6.0 M 2023-01-25T12:29:02.871 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 26 M 2023-01-25T12:29:02.871 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:02.886 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T12:29:02.965 INFO:teuthology.orchestra.run.smithi087.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.4 MB/s | 538 kB 00:00 2023-01-25T12:29:03.087 INFO:teuthology.orchestra.run.smithi087.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-01-25T12:29:03.134 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T12:29:03.316 INFO:teuthology.orchestra.run.smithi087.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 268 kB/s | 120 kB 00:00 2023-01-25T12:29:03.433 INFO:teuthology.orchestra.run.smithi087.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 175 kB/s | 60 kB 00:00 2023-01-25T12:29:03.491 INFO:teuthology.orchestra.run.smithi087.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 905 kB/s | 476 kB 00:00 2023-01-25T12:29:03.493 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:03.493 INFO:teuthology.orchestra.run.smithi087.stdout:Total 9.5 MB/s | 6.0 MB 00:00 2023-01-25T12:29:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T12:29:03.545 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:03.556 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:03.556 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:03.684 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:03.687 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:03.738 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T12:29:03.954 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T12:29:03.969 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:04.179 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T12:29:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T12:29:04.508 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T12:29:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T12:29:05.212 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T12:29:05.632 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T12:29:05.868 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T12:29:05.890 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T12:29:06.070 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:29:06.171 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T12:29:06.363 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T12:29:06.372 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:29:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T12:29:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T12:29:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T12:29:06.373 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T12:29:06.607 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T12:29:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:29:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T12:29:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:06.870 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T12:29:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T12:29:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T12:29:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T12:29:06.873 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:06.873 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:06.984 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-fuse 2023-01-25T12:29:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:29:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T12:29:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T12:29:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T12:29:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T12:29:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T12:29:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T12:29:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T12:29:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T12:29:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T12:29:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T12:29:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T12:29:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T12:29:07.695 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T12:29:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:07.761 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:07.761 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:07.761 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T12:29:07.761 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T12:29:07.761 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T12:29:07.762 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T12:29:07.762 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T12:29:07.762 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T12:29:07.762 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T12:29:07.762 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T12:29:07.763 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T12:29:07.763 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T12:29:07.763 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T12:29:07.763 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T12:29:07.763 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:07.764 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:07.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T12:29:08.663 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:08.842 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:08.843 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:08.843 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:29:08.843 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:08.844 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:08.844 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T12:29:08.844 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-01-25T12:29:08.844 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T12:29:08.844 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:08.845 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:08.845 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:08.845 INFO:teuthology.orchestra.run.smithi087.stdout:Install 2 Packages 2023-01-25T12:29:08.845 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:08.845 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 928 k 2023-01-25T12:29:08.846 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 2.9 M 2023-01-25T12:29:08.846 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:08.916 INFO:teuthology.orchestra.run.smithi087.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-25T12:29:09.132 INFO:teuthology.orchestra.run.smithi087.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 2.9 MB/s | 845 kB 00:00 2023-01-25T12:29:09.133 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:09.134 INFO:teuthology.orchestra.run.smithi087.stdout:Total 3.1 MB/s | 928 kB 00:00 2023-01-25T12:29:09.143 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:09.165 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:09.165 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:09.257 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:09.261 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:09.433 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:09.677 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T12:29:09.859 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:29:09.891 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:09.892 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:09.893 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:29:09.893 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:09.893 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:09.893 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T12:29:09.893 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:29:09.894 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T12:29:09.894 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T12:29:09.894 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T12:29:09.894 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T12:29:09.894 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:09.895 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:09.895 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:09.895 INFO:teuthology.orchestra.run.smithi067.stdout:Install 5 Packages 2023-01-25T12:29:09.895 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:09.896 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 6.0 M 2023-01-25T12:29:09.896 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 26 M 2023-01-25T12:29:09.896 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:09.985 INFO:teuthology.orchestra.run.smithi067.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.9 MB/s | 538 kB 00:00 2023-01-25T12:29:10.077 INFO:teuthology.orchestra.run.smithi067.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-01-25T12:29:10.102 INFO:teuthology.orchestra.run.smithi067.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 580 kB/s | 120 kB 00:00 2023-01-25T12:29:10.220 INFO:teuthology.orchestra.run.smithi067.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 425 kB/s | 60 kB 00:00 2023-01-25T12:29:10.268 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:29:10.268 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T12:29:10.378 INFO:teuthology.orchestra.run.smithi067.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.2 MB/s | 476 kB 00:00 2023-01-25T12:29:10.379 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:10.380 INFO:teuthology.orchestra.run.smithi067.stdout:Total 12 MB/s | 6.0 MB 00:00 2023-01-25T12:29:10.433 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:10.445 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:10.446 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:10.587 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:10.783 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T12:29:10.783 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:10.784 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:10.784 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T12:29:10.784 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:10.784 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:10.905 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:10.991 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-volume 2023-01-25T12:29:11.219 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T12:29:11.733 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:00 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:12.202 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T12:29:12.618 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T12:29:12.871 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:12.872 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-01-25T12:29:12.873 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:12.873 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T12:29:12.873 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:12.874 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:12.874 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:12.874 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:12.874 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:12.874 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 263 k 2023-01-25T12:29:12.875 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 1.3 M 2023-01-25T12:29:12.875 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:12.877 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T12:29:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:29:13.095 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-01-25T12:29:13.096 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:13.096 INFO:teuthology.orchestra.run.smithi087.stdout:Total 1.1 MB/s | 263 kB 00:00 2023-01-25T12:29:13.097 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:13.106 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:13.107 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:13.207 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:13.210 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:13.390 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:13.396 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:29:13.396 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T12:29:13.397 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T12:29:13.397 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T12:29:13.397 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T12:29:13.661 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:13.979 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T12:29:13.979 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:13.979 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:13.980 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:13.980 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T12:29:13.980 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T12:29:13.980 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T12:29:13.981 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T12:29:13.981 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:13.981 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:14.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-fuse 2023-01-25T12:29:14.104 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:14.547 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:14.548 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:14.548 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:14.548 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:14.751 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install librados-devel 2023-01-25T12:29:14.849 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:02 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:15.480 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:16.069 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:16.070 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:29:16.070 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:16.071 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T12:29:16.071 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-01-25T12:29:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T12:29:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:16.072 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:16.072 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:16.072 INFO:teuthology.orchestra.run.smithi067.stdout:Install 2 Packages 2023-01-25T12:29:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:16.073 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 928 k 2023-01-25T12:29:16.073 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 2.9 M 2023-01-25T12:29:16.073 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:16.152 INFO:teuthology.orchestra.run.smithi067.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2023-01-25T12:29:16.386 INFO:teuthology.orchestra.run.smithi067.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 2.6 MB/s | 845 kB 00:00 2023-01-25T12:29:16.386 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:16.387 INFO:teuthology.orchestra.run.smithi067.stdout:Total 2.9 MB/s | 928 kB 00:00 2023-01-25T12:29:16.397 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:16.420 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:16.420 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:16.522 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:16.526 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:16.631 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:16.632 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:16.632 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repo Size 2023-01-25T12:29:16.632 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:16.632 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:16.633 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T12:29:16.633 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:16.633 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:16.633 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:16.633 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:16.634 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:16.634 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 139 k 2023-01-25T12:29:16.634 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 512 k 2023-01-25T12:29:16.635 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:16.858 INFO:teuthology.orchestra.run.smithi087.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 619 kB/s | 139 kB 00:00 2023-01-25T12:29:16.858 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:16.859 INFO:teuthology.orchestra.run.smithi087.stdout:Total 616 kB/s | 139 kB 00:00 2023-01-25T12:29:16.859 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:16.864 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:16.864 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:16.917 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:16.920 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T12:29:17.050 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:29:17.215 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:17.522 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:17.640 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:29:17.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T12:29:18.002 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:18.003 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:18.003 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:18.003 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:18.003 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:18.004 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:18.178 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T12:29:18.179 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:18.179 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:18.179 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T12:29:18.179 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:18.179 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:18.258 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install libcephfs2 2023-01-25T12:29:18.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-volume 2023-01-25T12:29:18.982 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:07 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:19.157 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:07 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:20.034 INFO:teuthology.orchestra.run.smithi087.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:20.130 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:20.131 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:20.131 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:20.225 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install libcephfs-devel 2023-01-25T12:29:20.377 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:20.378 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:20.378 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-01-25T12:29:20.379 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:20.379 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:20.379 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T12:29:20.379 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:20.379 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:20.380 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:20.380 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:20.380 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:20.380 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 263 k 2023-01-25T12:29:20.381 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 1.3 M 2023-01-25T12:29:20.381 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:20.623 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 1.1 MB/s | 263 kB 00:00 2023-01-25T12:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:20.624 INFO:teuthology.orchestra.run.smithi067.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-25T12:29:20.625 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:20.636 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:20.637 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:20.737 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:20.741 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:20.935 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:20.941 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:09 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:21.681 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:22.084 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:22.084 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:22.084 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repo Size 2023-01-25T12:29:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:22.085 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T12:29:22.085 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:22.085 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:22.086 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:22.086 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 30 k 2023-01-25T12:29:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 137 k 2023-01-25T12:29:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:29:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:22.199 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:29:22.200 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:22.200 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:22.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librados-devel 2023-01-25T12:29:22.853 INFO:teuthology.orchestra.run.smithi087.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 40 kB/s | 30 kB 00:00 2023-01-25T12:29:22.853 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:22.854 INFO:teuthology.orchestra.run.smithi087.stdout:Total 40 kB/s | 30 kB 00:00 2023-01-25T12:29:22.854 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:22.857 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:22.857 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:22.891 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:22.956 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:23.140 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:23.197 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:11 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:23.421 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:23.849 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:23.849 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:23.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:23.850 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:23.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install librados2 2023-01-25T12:29:24.424 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:24.426 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:24.426 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2023-01-25T12:29:24.426 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:24.427 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T12:29:24.427 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:24.427 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:24.428 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:24.428 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:24.428 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:24.428 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 139 k 2023-01-25T12:29:24.428 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 512 k 2023-01-25T12:29:24.429 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:24.691 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:13 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:24.863 INFO:teuthology.orchestra.run.smithi067.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 318 kB/s | 139 kB 00:00 2023-01-25T12:29:24.864 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:24.864 INFO:teuthology.orchestra.run.smithi067.stdout:Total 317 kB/s | 139 kB 00:00 2023-01-25T12:29:24.865 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:24.932 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:24.935 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:25.268 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:25.726 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:25.826 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:25.827 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:25.827 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:25.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install librbd1 2023-01-25T12:29:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:26.116 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:26.117 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:26.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install libcephfs2 2023-01-25T12:29:26.649 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:27.158 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:27.688 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:27.788 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:27.789 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:27.790 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:27.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install python3-rados 2023-01-25T12:29:28.296 INFO:teuthology.orchestra.run.smithi067.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:28.408 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:28.410 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:28.410 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:28.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install libcephfs-devel 2023-01-25T12:29:28.722 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:29.263 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:29.762 INFO:teuthology.orchestra.run.smithi087.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:29.862 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:29.863 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:29.863 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:29.955 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install python3-rgw 2023-01-25T12:29:30.483 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:30.484 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:30.485 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2023-01-25T12:29:30.485 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:30.485 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:30.485 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T12:29:30.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:30.486 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:30.486 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:30.486 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:30.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:30.487 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 30 k 2023-01-25T12:29:30.487 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 137 k 2023-01-25T12:29:30.487 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:30.679 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:19 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:30.894 INFO:teuthology.orchestra.run.smithi067.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 74 kB/s | 30 kB 00:00 2023-01-25T12:29:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:Total 74 kB/s | 30 kB 00:00 2023-01-25T12:29:30.896 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:30.899 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:30.899 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:30.925 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:30.929 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:31.015 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:31.201 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:31.723 INFO:teuthology.orchestra.run.smithi087.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:31.822 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:31.823 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:31.824 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:31.923 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install python3-cephfs 2023-01-25T12:29:32.010 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:32.010 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:32.011 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:32.011 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:32.011 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:32.011 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:32.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librados2 2023-01-25T12:29:32.637 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:21 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:32.877 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:33.683 INFO:teuthology.orchestra.run.smithi087.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:33.784 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:33.785 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:33.786 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:33.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install python3-rbd 2023-01-25T12:29:33.986 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:34.099 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:34.100 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:34.100 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:34.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librbd1 2023-01-25T12:29:34.599 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:34.951 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:22 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:35.644 INFO:teuthology.orchestra.run.smithi087.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:35.745 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:35.746 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2023-01-25T12:29:35.747 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:35.840 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install rbd-fuse 2023-01-25T12:29:36.065 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:36.178 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:36.179 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:36.179 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:36.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rados 2023-01-25T12:29:36.567 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:37.039 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:37.706 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:37.707 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:37.707 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2023-01-25T12:29:37.708 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:37.708 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:37.708 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T12:29:37.708 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:37.709 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:37.709 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:37.709 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:37.709 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:37.710 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 94 k 2023-01-25T12:29:37.710 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 280 k 2023-01-25T12:29:37.710 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:37.886 INFO:teuthology.orchestra.run.smithi087.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 532 kB/s | 94 kB 00:00 2023-01-25T12:29:37.886 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:37.887 INFO:teuthology.orchestra.run.smithi087.stdout:Total 528 kB/s | 94 kB 00:00 2023-01-25T12:29:37.887 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:37.897 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:37.897 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:37.949 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:37.952 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:38.058 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:38.144 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:38.257 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:38.258 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:38.258 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:38.263 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:38.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rgw 2023-01-25T12:29:38.567 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:39.112 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:39.263 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:39.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:39.264 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:39.264 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:39.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:39.264 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:39.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install rbd-mirror 2023-01-25T12:29:40.186 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:40.260 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:40.372 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:40.373 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:40.374 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:40.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-cephfs 2023-01-25T12:29:41.221 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:41.327 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:41.328 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:41.328 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repo Size 2023-01-25T12:29:41.328 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:41.329 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:41.329 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T12:29:41.329 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:41.329 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:41.330 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:41.330 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:41.330 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:41.331 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 3.5 M 2023-01-25T12:29:41.331 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 16 M 2023-01-25T12:29:41.331 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:42.299 INFO:teuthology.orchestra.run.smithi087.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 3.6 MB/s | 3.5 MB 00:00 2023-01-25T12:29:42.300 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:42.300 INFO:teuthology.orchestra.run.smithi087.stdout:Total 3.6 MB/s | 3.5 MB 00:00 2023-01-25T12:29:42.300 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:42.312 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:42.313 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:42.330 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:42.376 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:42.379 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:42.443 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:42.444 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:42.444 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:42.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rbd 2023-01-25T12:29:42.842 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:43.016 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:43.290 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:43.450 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:43.944 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:43.945 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:43.945 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:43.945 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:43.945 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:43.946 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:44.159 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install rbd-nbd 2023-01-25T12:29:44.416 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:29:44.527 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:44.529 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-01-25T12:29:44.529 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:44.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-fuse 2023-01-25T12:29:44.890 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 12:27:11 PM UTC. 2023-01-25T12:29:45.409 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:46.031 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-01-25T12:29:46.032 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:46.032 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2023-01-25T12:29:46.032 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:46.032 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-01-25T12:29:46.033 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T12:29:46.033 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:46.033 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-01-25T12:29:46.033 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-01-25T12:29:46.034 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2023-01-25T12:29:46.034 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:46.034 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 180 k 2023-01-25T12:29:46.034 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 568 k 2023-01-25T12:29:46.034 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-01-25T12:29:46.269 INFO:teuthology.orchestra.run.smithi087.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 760 kB/s | 180 kB 00:00 2023-01-25T12:29:46.270 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:46.270 INFO:teuthology.orchestra.run.smithi087.stdout:Total 756 kB/s | 180 kB 00:00 2023-01-25T12:29:46.270 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-01-25T12:29:46.281 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-01-25T12:29:46.282 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-01-25T12:29:46.333 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-01-25T12:29:46.336 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-01-25T12:29:46.474 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-01-25T12:29:46.620 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:46.636 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:46.637 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:46.637 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2023-01-25T12:29:46.637 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:46.637 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:46.638 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T12:29:46.638 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:46.638 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:46.638 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:46.638 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:46.639 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:46.639 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 94 k 2023-01-25T12:29:46.640 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 280 k 2023-01-25T12:29:46.640 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:46.851 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 443 kB/s | 94 kB 00:00 2023-01-25T12:29:46.852 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:46.853 INFO:teuthology.orchestra.run.smithi067.stdout:Total 440 kB/s | 94 kB 00:00 2023-01-25T12:29:46.853 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:46.864 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:46.864 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:46.917 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:46.918 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:46.922 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:47.039 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:47.274 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:47.402 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:47.402 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-01-25T12:29:47.403 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:47.403 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-01-25T12:29:47.586 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:47.621 DEBUG:teuthology.parallel:result is None 2023-01-25T12:29:48.119 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:48.120 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:48.120 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:48.121 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:48.121 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:48.121 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:48.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-mirror 2023-01-25T12:29:49.086 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:37 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:50.306 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:50.307 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:50.308 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2023-01-25T12:29:50.308 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:50.308 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:50.308 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T12:29:50.309 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:50.309 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:50.309 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:50.309 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:50.309 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:50.310 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 3.5 M 2023-01-25T12:29:50.310 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 16 M 2023-01-25T12:29:50.310 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:50.971 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 5.2 MB/s | 3.5 MB 00:00 2023-01-25T12:29:50.973 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:50.973 INFO:teuthology.orchestra.run.smithi067.stdout:Total 5.2 MB/s | 3.5 MB 00:00 2023-01-25T12:29:50.974 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:50.986 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:50.987 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:51.055 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:51.058 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:51.693 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:52.219 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:52.761 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:52.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:52.762 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:52.762 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:52.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:52.762 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:52.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-nbd 2023-01-25T12:29:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:41 ago on Wed 25 Jan 2023 12:27:12 PM UTC. 2023-01-25T12:29:54.932 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-01-25T12:29:54.932 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:54.933 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2023-01-25T12:29:54.933 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:54.933 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-01-25T12:29:54.933 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T12:29:54.934 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:54.934 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-01-25T12:29:54.934 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-01-25T12:29:54.934 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2023-01-25T12:29:54.934 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:54.935 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 180 k 2023-01-25T12:29:54.935 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 568 k 2023-01-25T12:29:54.935 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-01-25T12:29:55.126 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 938 kB/s | 180 kB 00:00 2023-01-25T12:29:55.127 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:29:55.127 INFO:teuthology.orchestra.run.smithi067.stdout:Total 932 kB/s | 180 kB 00:00 2023-01-25T12:29:55.127 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-01-25T12:29:55.139 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-01-25T12:29:55.139 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-01-25T12:29:55.196 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-01-25T12:29:55.200 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-01-25T12:29:55.343 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-01-25T12:29:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:55.806 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:56.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:29:56.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:56.329 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-01-25T12:29:56.329 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:29:56.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:56.330 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-01-25T12:29:56.506 DEBUG:teuthology.parallel:result is None 2023-01-25T12:29:56.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo lsb_release -is 2023-01-25T12:29:56.546 INFO:teuthology.orchestra.run.smithi067.stdout:CentOSStream 2023-01-25T12:29:56.547 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:29:56.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:29:56.697 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T12:29:56.721 INFO:teuthology.orchestra.run.smithi067.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T12:29:56.722 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T12:29:56.722 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T12:29:56.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo lsb_release -is 2023-01-25T12:29:56.764 INFO:teuthology.orchestra.run.smithi087.stdout:CentOSStream 2023-01-25T12:29:56.765 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T12:29:56.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:29:56.919 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T12:29:56.942 INFO:teuthology.orchestra.run.smithi087.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T12:29:56.943 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T12:29:56.943 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T12:29:56.943 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T12:29:56.944 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:56.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T12:29:56.973 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:29:56.974 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T12:29:57.010 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T12:29:57.011 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:57.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T12:29:57.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T12:29:57.109 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:29:57.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T12:29:57.137 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T12:29:57.203 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T12:29:57.204 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:57.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T12:29:57.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T12:29:57.300 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:29:57.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T12:29:57.328 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T12:29:57.395 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T12:29:57.459 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T12:29:57.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2023-01-25T12:29:57.463 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2023-01-25T12:29:57.489 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T12:29:57.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T12:29:57.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T12:29:57.557 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T12:29:57.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T12:29:57.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T12:29:57.637 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T12:29:57.638 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T12:29:57.638 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137420', 'branch': 'wip-yuri6-testing-2023-01-24-0746', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7137420', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2023-01-24-0746'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri6-testing-2023-01-24-0746', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2023-01-24-0746', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'targets': {'smithi067.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfk+bkc/COlUouKAKfgLpBP1ZRmF2gypxXp8/D5fjmSlri4t++5qHNM2KPmv/HoJ+k+odgVbn5Dtb8U+B3cvDrT907m5x4vQhQ6G51IdlrEdMlofDgC6zdYCp310WTOjvl51THqr4r5tUtGLcSTj6MWn35+FxhXvJeBs/4fqwEVjgS9JeRQvvWodXSRTECwRAeY6T+EhrsQJl6W7UuM4OH3KXZfygeNuNBjhKmR3dQjMSuIu6l0G2lHKTPE9ih1KCw9fa+3htlPqgiJ06qMfAvg+nIAo9GT92GPAi8Nc+2ofuETous3Q8MTxBI6EPnx0OnHgEJ4jHqNXcXzfDFMazILe6hbDlXvIHgiKvSSEVJxFkPXW2K7mODtN4UnsPxlQ02ix36iC7VA4NR665EHsg2YJm8TcoUhwVqQW8VBaX70W2C6LMLNdffj0fjjhiOdeOtCuU1i60KBTXX/HVjGUqU6U43I7bKMv7OVr8ogscxeAQgx96GynY82qHPqt9I9xM=', 'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOO/Pj+7iKcD4i8uJKpXzqNEfrKvFYOp8XUDl6AcwxPQswLjdAFYX5fXlkB4GsKQJi6YyM6TX6ejJ1mZ2SeJzPc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache readproxy', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:27:30', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T12:29:57.639 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T12:29:57.667 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T12:29:57.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T12:29:57.725 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T12:29:57.753 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:57.754 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T12:29:57.810 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T12:29:57.810 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2023-01-25T12:29:57.868 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T12:29:57.869 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:57.869 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 362393 Links: 1 2023-01-25T12:29:57.869 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:57.869 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:57.870 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-01-25 12:29:51.937336928 +0000 2023-01-25T12:29:57.870 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-01-25 12:26:24.016939321 +0000 2023-01-25T12:29:57.870 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-01-25 12:26:24.016939321 +0000 2023-01-25T12:29:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-01-25T12:29:57.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T12:29:57.935 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-01-25T12:29:57.935 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-01-25T12:29:57.936 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000233856 s, 2.2 MB/s 2023-01-25T12:29:57.937 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T12:29:57.996 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2023-01-25T12:29:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T12:29:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:58.055 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 366553 Links: 1 2023-01-25T12:29:58.055 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:58.056 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:58.056 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-01-25 12:29:51.937336928 +0000 2023-01-25T12:29:58.056 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-01-25 12:26:24.563928040 +0000 2023-01-25T12:29:58.056 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-01-25 12:26:24.563928040 +0000 2023-01-25T12:29:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-01-25T12:29:58.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T12:29:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-01-25T12:29:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-01-25T12:29:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000281888 s, 1.8 MB/s 2023-01-25T12:29:58.124 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T12:29:58.184 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2023-01-25T12:29:58.242 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T12:29:58.242 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:58.242 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 362455 Links: 1 2023-01-25T12:29:58.243 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:58.243 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:58.244 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-01-25 12:29:51.937336928 +0000 2023-01-25T12:29:58.244 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-01-25 12:26:25.056917872 +0000 2023-01-25T12:29:58.244 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-01-25 12:26:25.056917872 +0000 2023-01-25T12:29:58.244 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-01-25T12:29:58.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T12:29:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-01-25T12:29:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-01-25T12:29:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000210186 s, 2.4 MB/s 2023-01-25T12:29:58.311 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T12:29:58.371 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2023-01-25T12:29:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T12:29:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:58.430 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 377982 Links: 1 2023-01-25T12:29:58.430 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:58.430 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:58.430 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-01-25 12:29:51.937336928 +0000 2023-01-25T12:29:58.430 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-01-25 12:26:25.570907271 +0000 2023-01-25T12:29:58.431 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-01-25 12:26:25.570907271 +0000 2023-01-25T12:29:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-01-25T12:29:58.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T12:29:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-01-25T12:29:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-01-25T12:29:58.496 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000203048 s, 2.5 MB/s 2023-01-25T12:29:58.497 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T12:29:58.556 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T12:29:58.557 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:29:58.557 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T12:29:58.576 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T12:29:58.576 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2023-01-25T12:29:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T12:29:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:58.634 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 369474 Links: 1 2023-01-25T12:29:58.635 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:58.635 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:58.635 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-01-25 12:29:43.194258517 +0000 2023-01-25T12:29:58.635 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-01-25 12:26:14.332987185 +0000 2023-01-25T12:29:58.635 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-01-25 12:26:14.332987185 +0000 2023-01-25T12:29:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-01-25T12:29:58.636 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T12:29:58.704 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-01-25T12:29:58.705 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-01-25T12:29:58.705 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000210178 s, 2.4 MB/s 2023-01-25T12:29:58.705 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T12:29:58.764 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2023-01-25T12:29:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T12:29:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:58.822 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 373110 Links: 1 2023-01-25T12:29:58.822 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:58.823 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:58.823 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-01-25 12:29:43.194258517 +0000 2023-01-25T12:29:58.823 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-01-25 12:26:14.838975904 +0000 2023-01-25T12:29:58.823 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-01-25 12:26:14.838975904 +0000 2023-01-25T12:29:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-01-25T12:29:58.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T12:29:58.887 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-01-25T12:29:58.888 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-01-25T12:29:58.888 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00023887 s, 2.1 MB/s 2023-01-25T12:29:58.889 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T12:29:58.948 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2023-01-25T12:29:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T12:29:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:59.007 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 364286 Links: 1 2023-01-25T12:29:59.007 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:59.008 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:59.008 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-01-25 12:29:43.194258517 +0000 2023-01-25T12:29:59.008 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-01-25 12:26:15.357964333 +0000 2023-01-25T12:29:59.008 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-01-25 12:26:15.357964333 +0000 2023-01-25T12:29:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-01-25T12:29:59.009 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T12:29:59.072 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-01-25T12:29:59.073 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-01-25T12:29:59.073 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000187976 s, 2.7 MB/s 2023-01-25T12:29:59.074 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T12:29:59.133 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2023-01-25T12:29:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T12:29:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T12:29:59.192 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 372385 Links: 1 2023-01-25T12:29:59.192 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T12:29:59.193 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T12:29:59.193 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-01-25 12:29:43.194258517 +0000 2023-01-25T12:29:59.193 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-01-25 12:26:15.863953052 +0000 2023-01-25T12:29:59.193 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-01-25 12:26:15.863953052 +0000 2023-01-25T12:29:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-01-25T12:29:59.194 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T12:29:59.257 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-01-25T12:29:59.259 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-01-25T12:29:59.260 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000206958 s, 2.5 MB/s 2023-01-25T12:29:59.262 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T12:29:59.320 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T12:29:59.321 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T12:29:59.321 INFO:tasks.ceph:Generating config... 2023-01-25T12:29:59.322 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T12:29:59.322 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2023-01-25T12:29:59.323 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2023-01-25T12:29:59.323 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-25T12:29:59.323 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-25T12:29:59.324 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T12:29:59.324 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-25T12:29:59.324 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-25T12:29:59.324 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T12:29:59.325 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T12:29:59.325 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-25T12:29:59.325 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-25T12:29:59.325 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-25T12:29:59.326 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-25T12:29:59.326 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-25T12:29:59.326 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T12:29:59.326 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-01-25T12:29:59.327 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T12:29:59.327 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T12:29:59.327 INFO:tasks.ceph:[osd] osd delete sleep = 1 2023-01-25T12:29:59.327 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-25T12:29:59.328 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-01-25T12:29:59.328 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T12:29:59.328 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-01-25T12:29:59.328 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T12:29:59.329 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-25T12:29:59.329 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T12:29:59.329 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T12:29:59.330 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-25T12:29:59.330 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-25T12:29:59.330 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T12:29:59.330 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-01-25T12:29:59.331 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T12:29:59.331 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T12:29:59.331 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T12:29:59.331 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-25T12:29:59.332 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-25T12:29:59.332 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-25T12:29:59.332 INFO:tasks.ceph:[global] ms inject delay max = 1 2023-01-25T12:29:59.333 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2023-01-25T12:29:59.333 INFO:tasks.ceph:[global] ms inject delay type = osd 2023-01-25T12:29:59.333 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2023-01-25T12:29:59.333 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-25T12:29:59.334 INFO:tasks.ceph:[global] ms type = async 2023-01-25T12:29:59.334 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-25T12:29:59.334 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-25T12:29:59.334 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-25T12:29:59.335 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T12:29:59.335 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-25T12:29:59.335 INFO:tasks.ceph:[global] osd_recovery_max_active = 10 2023-01-25T12:29:59.335 INFO:tasks.ceph:[global] osd_recovery_max_single_start = 10 2023-01-25T12:29:59.336 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T12:29:59.336 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T12:29:59.336 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T12:29:59.337 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T12:29:59.337 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T12:29:59.337 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-25T12:29:59.337 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-25T12:29:59.338 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-25T12:29:59.338 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-25T12:29:59.338 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T12:29:59.338 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-25T12:29:59.339 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T12:29:59.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T12:29:59.392 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T12:29:59.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T12:29:59.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T12:29:59.471 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.c', '[v2:172.21.15.67:3301,v1:172.21.15.67:6790]'), ('mon.b', '172.21.15.87')] 2023-01-25T12:29:59.472 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],172.21.15.87', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T12:29:59.473 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:59.473 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T12:29:59.529 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --addv c '[v2:172.21.15.67:3301,v1:172.21.15.67:6790]' --add b 172.21.15.87 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:29:59.620 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:29:59.620 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid b623256f-e4e6-420d-b240-c3bfae6a45f5 2023-01-25T12:29:59.621 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = pacific 2023-01-25T12:29:59.621 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2023-01-25T12:29:59.621 INFO:teuthology.orchestra.run.smithi067.stdout:fsid b623256f-e4e6-420d-b240-c3bfae6a45f5 2023-01-25T12:29:59.622 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2023-01-25T12:29:59.619311+0000 2023-01-25T12:29:59.622 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-01-25T12:29:59.619311+0000 2023-01-25T12:29:59.623 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 16 (pacific) 2023-01-25T12:29:59.623 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 3 2023-01-25T12:29:59.623 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2023-01-25T12:29:59.623 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.b 2023-01-25T12:29:59.624 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] mon.c 2023-01-25T12:29:59.624 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T12:29:59.626 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T12:29:59.644 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b623256f-e4e6-420d-b240-c3bfae6a45f5... 2023-01-25T12:29:59.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T12:29:59.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T12:29:59.731 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2023-01-25T12:29:59.731 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2023-01-25T12:29:59.732 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T12:29:59.732 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2023-01-25T12:29:59.732 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.732 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2023-01-25T12:29:59.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.734 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2023-01-25T12:29:59.734 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2023-01-25T12:29:59.734 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.734 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T12:29:59.735 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2023-01-25T12:29:59.735 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.735 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T12:29:59.735 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T12:29:59.736 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.736 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2023-01-25T12:29:59.736 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.737 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2023-01-25T12:29:59.737 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2023-01-25T12:29:59.737 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2023-01-25T12:29:59.737 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2023-01-25T12:29:59.737 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2023-01-25T12:29:59.738 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2023-01-25T12:29:59.738 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2023-01-25T12:29:59.738 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T12:29:59.738 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T12:29:59.739 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2023-01-25T12:29:59.739 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.739 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T12:29:59.739 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.740 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2023-01-25T12:29:59.740 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.740 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2023-01-25T12:29:59.741 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.741 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2023-01-25T12:29:59.741 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2023-01-25T12:29:59.741 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2023-01-25T12:29:59.741 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],172.21.15.87" 2023-01-25T12:29:59.742 INFO:teuthology.orchestra.run.smithi067.stdout: mon client directed command retry = 5 2023-01-25T12:29:59.742 INFO:teuthology.orchestra.run.smithi067.stdout: mon election default strategy = 3 2023-01-25T12:29:59.742 INFO:teuthology.orchestra.run.smithi067.stdout: ms bind msgr1 = False 2023-01-25T12:29:59.742 INFO:teuthology.orchestra.run.smithi067.stdout: ms bind msgr2 = True 2023-01-25T12:29:59.742 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay max = 1 2023-01-25T12:29:59.743 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay probability = 0.005 2023-01-25T12:29:59.743 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay type = osd 2023-01-25T12:29:59.743 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject internal delays = 0.002 2023-01-25T12:29:59.743 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject socket failures = 2500 2023-01-25T12:29:59.743 INFO:teuthology.orchestra.run.smithi067.stdout: ms type = async 2023-01-25T12:29:59.744 INFO:teuthology.orchestra.run.smithi067.stdout: osd_max_pg_log_entries = 2 2023-01-25T12:29:59.744 INFO:teuthology.orchestra.run.smithi067.stdout: osd_min_pg_log_entries = 1 2023-01-25T12:29:59.744 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pg_log_trim_min = 0 2023-01-25T12:29:59.744 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_min_size = 2 2023-01-25T12:29:59.744 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_size = 3 2023-01-25T12:29:59.745 INFO:teuthology.orchestra.run.smithi067.stdout: osd_recovery_max_active = 10 2023-01-25T12:29:59.745 INFO:teuthology.orchestra.run.smithi067.stdout: osd_recovery_max_single_start = 10 2023-01-25T12:29:59.745 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = b623256f-e4e6-420d-b240-c3bfae6a45f5 2023-01-25T12:29:59.745 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.746 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2023-01-25T12:29:59.746 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2023-01-25T12:29:59.746 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.746 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2023-01-25T12:29:59.747 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 120 2023-01-25T12:29:59.747 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.747 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2023-01-25T12:29:59.747 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2023-01-25T12:29:59.748 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.748 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2023-01-25T12:29:59.748 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2023-01-25T12:29:59.748 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2023-01-25T12:29:59.748 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.749 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2023-01-25T12:29:59.749 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2023-01-25T12:29:59.749 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.749 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2023-01-25T12:29:59.749 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2023-01-25T12:29:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2023-01-25T12:29:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2023-01-25T12:29:59.751 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2023-01-25T12:29:59.751 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.751 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2023-01-25T12:29:59.751 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2023-01-25T12:29:59.751 INFO:teuthology.orchestra.run.smithi067.stdout: debug monc = 20 2023-01-25T12:29:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: bluefs allocator = hybrid 2023-01-25T12:29:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore allocator = hybrid 2023-01-25T12:29:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2023-01-25T12:29:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2023-01-25T12:29:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore zero block detection = True 2023-01-25T12:29:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 20 2023-01-25T12:29:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 20 2023-01-25T12:29:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-01-25T12:29:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 10 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: osd backoff on peering = True 2023-01-25T12:29:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: osd blocked scrub grace period = 3600 2023-01-25T12:29:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug reject backfill probability = 0.3 2023-01-25T12:29:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify cached snaps = True 2023-01-25T12:29:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify missing on start = True 2023-01-25T12:29:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: osd delete sleep = 1 2023-01-25T12:29:59.756 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2023-01-25T12:29:59.756 INFO:teuthology.orchestra.run.smithi067.stdout: osd max backfills = 3 2023-01-25T12:29:59.756 INFO:teuthology.orchestra.run.smithi067.stdout: osd max markdown count = 1000 2023-01-25T12:29:59.756 INFO:teuthology.orchestra.run.smithi067.stdout: osd max scrubs = 3 2023-01-25T12:29:59.757 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock override recovery settings = True 2023-01-25T12:29:59.757 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2023-01-25T12:29:59.757 INFO:teuthology.orchestra.run.smithi067.stdout: osd op queue = debug_random 2023-01-25T12:29:59.757 INFO:teuthology.orchestra.run.smithi067.stdout: osd op queue cut off = debug_random 2023-01-25T12:29:59.758 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub min interval = 60 2023-01-25T12:29:59.758 INFO:teuthology.orchestra.run.smithi067.stdout: osd shutdown pgref assert = True 2023-01-25T12:29:59.758 INFO:teuthology.orchestra.run.smithi067.stdout: osd snap trim sleep = 2 2023-01-25T12:29:59.758 INFO:teuthology.orchestra.run.smithi067.stdout: osd_max_pg_log_entries = 3000 2023-01-25T12:29:59.759 INFO:teuthology.orchestra.run.smithi067.stdout: osd_min_pg_log_entries = 3000 2023-01-25T12:29:59.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.759 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2023-01-25T12:29:59.759 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-01-25T12:29:59.759 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2023-01-25T12:29:59.760 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-01-25T12:29:59.760 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-01-25T12:29:59.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-01-25T12:29:59.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-01-25T12:29:59.760 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2023-01-25T12:29:59.761 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.761 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2023-01-25T12:29:59.761 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-01-25T12:29:59.761 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-01-25T12:29:59.762 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2023-01-25T12:29:59.762 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-01-25T12:29:59.762 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2023-01-25T12:29:59.762 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2023-01-25T12:29:59.762 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-01-25T12:29:59.763 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2023-01-25T12:29:59.763 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2023-01-25T12:29:59.763 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-01-25T12:29:59.763 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.763 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T12:29:59.764 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T12:29:59.764 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T12:29:59.764 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.764 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T12:29:59.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T12:29:59.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T12:29:59.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.765 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2023-01-25T12:29:59.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2023-01-25T12:29:59.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2023-01-25T12:29:59.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon min osdmap epochs = 50 2023-01-25T12:29:59.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon osdmap full prune interval = 2 2023-01-25T12:29:59.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon osdmap full prune min = 15 2023-01-25T12:29:59.767 INFO:teuthology.orchestra.run.smithi067.stdout: mon osdmap full prune txsize = 2 2023-01-25T12:29:59.767 INFO:teuthology.orchestra.run.smithi067.stdout: mon scrub interval = 300 2023-01-25T12:29:59.768 INFO:teuthology.orchestra.run.smithi067.stdout: paxos service trim min = 10 2023-01-25T12:29:59.768 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:29:59.768 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2023-01-25T12:29:59.768 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2023-01-25T12:29:59.769 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2023-01-25T12:29:59.769 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2023-01-25T12:29:59.769 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T12:29:59.769 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T12:29:59.769 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2023-01-25T12:29:59.770 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2023-01-25T12:29:59.770 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2023-01-25T12:29:59.772 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2023-01-25T12:29:59.772 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2023-01-25T12:29:59.772 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T12:29:59.772 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2023-01-25T12:29:59.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.773 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2023-01-25T12:29:59.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.773 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2023-01-25T12:29:59.773 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.774 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2023-01-25T12:29:59.774 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2023-01-25T12:29:59.774 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.774 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2023-01-25T12:29:59.774 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2023-01-25T12:29:59.775 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.775 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T12:29:59.775 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2023-01-25T12:29:59.775 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.775 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T12:29:59.776 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T12:29:59.776 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.776 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2023-01-25T12:29:59.776 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.776 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2023-01-25T12:29:59.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2023-01-25T12:29:59.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2023-01-25T12:29:59.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2023-01-25T12:29:59.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2023-01-25T12:29:59.777 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2023-01-25T12:29:59.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2023-01-25T12:29:59.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T12:29:59.778 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T12:29:59.779 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2023-01-25T12:29:59.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.779 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T12:29:59.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.779 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2023-01-25T12:29:59.780 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.780 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2023-01-25T12:29:59.780 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.780 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2023-01-25T12:29:59.780 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2023-01-25T12:29:59.781 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2023-01-25T12:29:59.781 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],172.21.15.87" 2023-01-25T12:29:59.781 INFO:teuthology.orchestra.run.smithi087.stdout: mon client directed command retry = 5 2023-01-25T12:29:59.781 INFO:teuthology.orchestra.run.smithi087.stdout: mon election default strategy = 3 2023-01-25T12:29:59.781 INFO:teuthology.orchestra.run.smithi087.stdout: ms bind msgr1 = False 2023-01-25T12:29:59.782 INFO:teuthology.orchestra.run.smithi087.stdout: ms bind msgr2 = True 2023-01-25T12:29:59.782 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject delay max = 1 2023-01-25T12:29:59.782 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject delay probability = 0.005 2023-01-25T12:29:59.782 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject delay type = osd 2023-01-25T12:29:59.783 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject internal delays = 0.002 2023-01-25T12:29:59.783 INFO:teuthology.orchestra.run.smithi087.stdout: ms inject socket failures = 2500 2023-01-25T12:29:59.783 INFO:teuthology.orchestra.run.smithi087.stdout: ms type = async 2023-01-25T12:29:59.783 INFO:teuthology.orchestra.run.smithi087.stdout: osd_max_pg_log_entries = 2 2023-01-25T12:29:59.783 INFO:teuthology.orchestra.run.smithi087.stdout: osd_min_pg_log_entries = 1 2023-01-25T12:29:59.784 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pg_log_trim_min = 0 2023-01-25T12:29:59.784 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_min_size = 2 2023-01-25T12:29:59.784 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_size = 3 2023-01-25T12:29:59.784 INFO:teuthology.orchestra.run.smithi087.stdout: osd_recovery_max_active = 10 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout: osd_recovery_max_single_start = 10 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = b623256f-e4e6-420d-b240-c3bfae6a45f5 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2023-01-25T12:29:59.785 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.786 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2023-01-25T12:29:59.786 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 120 2023-01-25T12:29:59.786 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.786 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2023-01-25T12:29:59.786 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2023-01-25T12:29:59.787 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.787 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2023-01-25T12:29:59.787 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2023-01-25T12:29:59.787 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2023-01-25T12:29:59.788 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.788 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2023-01-25T12:29:59.788 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2023-01-25T12:29:59.788 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.789 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2023-01-25T12:29:59.789 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.789 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2023-01-25T12:29:59.789 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.790 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2023-01-25T12:29:59.790 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.790 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2023-01-25T12:29:59.790 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2023-01-25T12:29:59.790 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.791 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2023-01-25T12:29:59.791 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2023-01-25T12:29:59.791 INFO:teuthology.orchestra.run.smithi087.stdout: debug monc = 20 2023-01-25T12:29:59.791 INFO:teuthology.orchestra.run.smithi087.stdout: bluefs allocator = hybrid 2023-01-25T12:29:59.791 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = hybrid 2023-01-25T12:29:59.792 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2023-01-25T12:29:59.792 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2023-01-25T12:29:59.792 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore zero block detection = True 2023-01-25T12:29:59.792 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 20 2023-01-25T12:29:59.793 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 20 2023-01-25T12:29:59.793 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2023-01-25T12:29:59.793 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2023-01-25T12:29:59.793 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 10 2023-01-25T12:29:59.794 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T12:29:59.794 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2023-01-25T12:29:59.794 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2023-01-25T12:29:59.794 INFO:teuthology.orchestra.run.smithi087.stdout: osd backoff on peering = True 2023-01-25T12:29:59.794 INFO:teuthology.orchestra.run.smithi087.stdout: osd blocked scrub grace period = 3600 2023-01-25T12:29:59.795 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug reject backfill probability = 0.3 2023-01-25T12:29:59.795 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify cached snaps = True 2023-01-25T12:29:59.795 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify missing on start = True 2023-01-25T12:29:59.795 INFO:teuthology.orchestra.run.smithi087.stdout: osd delete sleep = 1 2023-01-25T12:29:59.796 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2023-01-25T12:29:59.796 INFO:teuthology.orchestra.run.smithi087.stdout: osd max backfills = 3 2023-01-25T12:29:59.796 INFO:teuthology.orchestra.run.smithi087.stdout: osd max markdown count = 1000 2023-01-25T12:29:59.796 INFO:teuthology.orchestra.run.smithi087.stdout: osd max scrubs = 3 2023-01-25T12:29:59.796 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock override recovery settings = True 2023-01-25T12:29:59.797 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2023-01-25T12:29:59.797 INFO:teuthology.orchestra.run.smithi087.stdout: osd op queue = debug_random 2023-01-25T12:29:59.797 INFO:teuthology.orchestra.run.smithi087.stdout: osd op queue cut off = debug_random 2023-01-25T12:29:59.797 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub min interval = 60 2023-01-25T12:29:59.798 INFO:teuthology.orchestra.run.smithi087.stdout: osd shutdown pgref assert = True 2023-01-25T12:29:59.798 INFO:teuthology.orchestra.run.smithi087.stdout: osd snap trim sleep = 2 2023-01-25T12:29:59.798 INFO:teuthology.orchestra.run.smithi087.stdout: osd_max_pg_log_entries = 3000 2023-01-25T12:29:59.798 INFO:teuthology.orchestra.run.smithi087.stdout: osd_min_pg_log_entries = 3000 2023-01-25T12:29:59.798 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.799 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2023-01-25T12:29:59.799 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2023-01-25T12:29:59.799 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2023-01-25T12:29:59.800 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2023-01-25T12:29:59.800 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2023-01-25T12:29:59.800 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2023-01-25T12:29:59.800 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2023-01-25T12:29:59.800 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2023-01-25T12:29:59.801 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.801 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2023-01-25T12:29:59.801 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2023-01-25T12:29:59.801 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2023-01-25T12:29:59.801 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2023-01-25T12:29:59.802 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2023-01-25T12:29:59.802 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2023-01-25T12:29:59.802 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2023-01-25T12:29:59.802 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2023-01-25T12:29:59.802 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2023-01-25T12:29:59.803 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2023-01-25T12:29:59.803 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2023-01-25T12:29:59.803 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.803 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T12:29:59.804 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T12:29:59.804 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T12:29:59.804 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.804 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T12:29:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T12:29:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T12:29:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2023-01-25T12:29:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2023-01-25T12:29:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2023-01-25T12:29:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: mon min osdmap epochs = 50 2023-01-25T12:29:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: mon osdmap full prune interval = 2 2023-01-25T12:29:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: mon osdmap full prune min = 15 2023-01-25T12:29:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: mon osdmap full prune txsize = 2 2023-01-25T12:29:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: mon scrub interval = 300 2023-01-25T12:29:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: paxos service trim min = 10 2023-01-25T12:29:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-01-25T12:29:59.808 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2023-01-25T12:29:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2023-01-25T12:29:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2023-01-25T12:29:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2023-01-25T12:29:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T12:29:59.809 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T12:29:59.809 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2023-01-25T12:29:59.809 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2023-01-25T12:29:59.809 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2023-01-25T12:29:59.810 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T12:29:59.810 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T12:29:59.870 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T12:29:59.871 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:59.871 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T12:29:59.889 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:59.889 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T12:29:59.947 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:29:59.948 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:29:59.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T12:29:59.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T12:30:00.027 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:00.027 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:00.086 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:30:00.086 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:00.086 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T12:30:00.087 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T12:30:00.126 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:00.127 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:00.185 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T12:30:00.186 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T12:30:00.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T12:30:00.242 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T12:30:00.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T12:30:00.307 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T12:30:00.310 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T12:30:00.310 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T12:30:00.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T12:30:00.356 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T12:30:00.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T12:30:00.413 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T12:30:00.426 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T12:30:00.426 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T12:30:00.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T12:30:00.455 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:00.455 INFO:tasks.ceph:role: osd.0 2023-01-25T12:30:00.455 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:30:00.456 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T12:30:00.531 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:00.531 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:00.532 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:00.532 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:00.532 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:00.532 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:00.532 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:00.533 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:00.533 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:00.533 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-01-25T12:30:00.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-01-25T12:30:00.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T12:30:01.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T12:30:01.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T12:30:01.244 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:01.244 INFO:tasks.ceph:role: osd.1 2023-01-25T12:30:01.244 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:30:01.245 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T12:30:01.320 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:01.320 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:01.320 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:01.321 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:01.321 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:01.321 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:01.321 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:01.322 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:01.322 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:01.322 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:01.748 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-01-25T12:30:01.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-01-25T12:30:01.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T12:30:01.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T12:30:01.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T12:30:01.959 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:01.959 INFO:tasks.ceph:role: osd.2 2023-01-25T12:30:01.960 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:30:01.960 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T12:30:02.035 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:02.036 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:02.036 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:02.036 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:02.036 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:02.037 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:02.037 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:02.483 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-01-25T12:30:02.484 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-01-25T12:30:02.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T12:30:02.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T12:30:02.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T12:30:02.745 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:02.745 INFO:tasks.ceph:role: osd.3 2023-01-25T12:30:02.745 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com 2023-01-25T12:30:02.746 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T12:30:02.822 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:02.822 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:02.822 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:02.823 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:02.823 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:02.824 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:02.824 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:03.269 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-01-25T12:30:03.270 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-01-25T12:30:03.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T12:30:03.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T12:30:03.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:03.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.581+0000 7f4c22d79580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T12:30:03.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.581+0000 7f4c22d79580 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T12:30:03.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.581+0000 7f4c22d79580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T12:30:03.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.581+0000 7f4c22d79580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T12:30:03.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.581+0000 7f4c22d79580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T12:30:03.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:03.582+0000 7f4c22d79580 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T12:30:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:04.107+0000 7f4c22d79580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:04.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:04.870+0000 7f4c22d79580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:05.632+0000 7f4c22d79580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:06.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T12:30:06.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.551+0000 7fe57e465580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T12:30:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.551+0000 7fe57e465580 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T12:30:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.551+0000 7fe57e465580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T12:30:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.551+0000 7fe57e465580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T12:30:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.551+0000 7fe57e465580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T12:30:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:06.552+0000 7fe57e465580 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T12:30:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:07.075+0000 7fe57e465580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:07.837+0000 7fe57e465580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:08.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:08.598+0000 7fe57e465580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:09.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T12:30:09.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.513+0000 7f7a1da88580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T12:30:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.513+0000 7f7a1da88580 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T12:30:09.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.513+0000 7f7a1da88580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T12:30:09.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.513+0000 7f7a1da88580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T12:30:09.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.513+0000 7f7a1da88580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T12:30:09.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:09.514+0000 7f7a1da88580 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T12:30:10.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:10.039+0000 7f7a1da88580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:10.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:10.804+0000 7f7a1da88580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:11.566+0000 7f7a1da88580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:12.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T12:30:12.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:12.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T12:30:12.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T12:30:12.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T12:30:12.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T12:30:12.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T12:30:12.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:12.489+0000 7f829cb08580 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T12:30:13.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:13.013+0000 7f829cb08580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:13.779+0000 7f829cb08580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-01-25T12:30:14.540+0000 7f829cb08580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:15.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T12:30:15.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T12:30:15.404 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:15.404 INFO:tasks.ceph:role: osd.4 2023-01-25T12:30:15.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:30:15.405 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T12:30:15.482 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:15.482 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:15.483 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:15.483 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:15.484 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:15.484 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:15.484 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:15.484 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:15.914 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2023-01-25T12:30:15.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2023-01-25T12:30:15.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T12:30:16.098 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T12:30:16.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T12:30:16.205 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:16.206 INFO:tasks.ceph:role: osd.5 2023-01-25T12:30:16.206 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:30:16.206 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T12:30:16.281 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:16.282 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:16.282 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:16.282 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:16.283 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:16.283 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:16.283 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:16.283 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:16.284 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:16.284 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:16.704 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2023-01-25T12:30:16.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2023-01-25T12:30:16.706 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T12:30:16.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T12:30:16.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T12:30:16.917 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:16.918 INFO:tasks.ceph:role: osd.6 2023-01-25T12:30:16.919 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:30:16.919 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T12:30:16.992 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:16.993 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:16.993 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:16.993 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:16.994 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:16.994 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:16.994 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:16.994 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:16.994 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:16.995 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:17.442 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2023-01-25T12:30:17.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2023-01-25T12:30:17.444 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T12:30:17.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T12:30:17.647 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T12:30:17.714 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T12:30:17.715 INFO:tasks.ceph:role: osd.7 2023-01-25T12:30:17.715 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2023-01-25T12:30:17.715 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T12:30:17.789 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T12:30:17.790 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T12:30:17.790 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T12:30:17.790 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T12:30:17.790 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T12:30:17.790 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2023-01-25T12:30:17.791 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T12:30:17.791 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T12:30:17.791 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T12:30:17.791 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T12:30:18.236 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2023-01-25T12:30:18.237 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2023-01-25T12:30:18.237 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T12:30:18.426 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T12:30:18.460 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:18.561 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.557+0000 7fe854070580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T12:30:18.562 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.557+0000 7fe854070580 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T12:30:18.562 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.557+0000 7fe854070580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T12:30:18.563 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.557+0000 7fe854070580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T12:30:18.563 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.557+0000 7fe854070580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T12:30:18.563 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:18.558+0000 7fe854070580 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T12:30:19.086 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:19.084+0000 7fe854070580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:19.848 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:19.846+0000 7fe854070580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:20.615 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:20.612+0000 7fe854070580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:21.407 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T12:30:21.437 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:21.527 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.525+0000 7fd51f8d2580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T12:30:21.528 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.525+0000 7fd51f8d2580 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T12:30:21.528 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.525+0000 7fd51f8d2580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T12:30:21.528 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.525+0000 7fd51f8d2580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T12:30:21.528 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.525+0000 7fd51f8d2580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T12:30:21.529 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:21.526+0000 7fd51f8d2580 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T12:30:22.051 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:22.049+0000 7fd51f8d2580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:22.812 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:22.810+0000 7fd51f8d2580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:23.578 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:23.576+0000 7fd51f8d2580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:24.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T12:30:24.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:24.482 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T12:30:24.482 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T12:30:24.483 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T12:30:24.483 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T12:30:24.483 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T12:30:24.483 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:24.480+0000 7efe8d820580 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T12:30:25.006 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:25.004+0000 7efe8d820580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:25.774 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:25.773+0000 7efe8d820580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:26.535 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:26.534+0000 7efe8d820580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:27.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T12:30:27.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:27.446 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.444+0000 7fd170623580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T12:30:27.446 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.444+0000 7fd170623580 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T12:30:27.446 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.444+0000 7fd170623580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T12:30:27.447 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.444+0000 7fd170623580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T12:30:27.447 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.444+0000 7fd170623580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T12:30:27.447 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.445+0000 7fd170623580 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T12:30:27.975 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:27.975+0000 7fd170623580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:28.736 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:28.735+0000 7fd170623580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:29.499 INFO:teuthology.orchestra.run.smithi087.stderr:2023-01-25T12:30:29.498+0000 7fd170623580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T12:30:30.289 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T12:30:30.319 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T12:30:30.319 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T12:30:30.349 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T12:30:30.418 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T12:30:30.487 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T12:30:30.556 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T12:30:30.625 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.625 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T12:30:30.655 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T12:30:30.723 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.724 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T12:30:30.792 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.793 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T12:30:30.859 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T12:30:30.927 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:30.927 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T12:30:30.946 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:30.947 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T12:30:30.984 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T12:30:30.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T12:30:30.988 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T12:30:31.027 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2023-01-25T12:30:31.028 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDIINFjYPtmDhAA1jz+hQrZJV23miXXmBCHQg== 2023-01-25T12:30:31.028 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2023-01-25T12:30:31.028 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDLINFj4m67IhAAjy+R2Z3A3nZrF7LdgjESng== 2023-01-25T12:30:31.028 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2023-01-25T12:30:31.029 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDOINFjnBXvIBAAloncHGYjSJHgZLbPrEMMaA== 2023-01-25T12:30:31.029 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2023-01-25T12:30:31.030 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDRINFjPrqoHhAAhA1NvjF8PsKR7qeRwskB6Q== 2023-01-25T12:30:31.030 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.3] 2023-01-25T12:30:31.030 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDUINFjbXg1HRAAoFTJYnp+vLxj1mDre9tpjQ== 2023-01-25T12:30:31.031 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2023-01-25T12:30:31.031 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDIINFjE3tLEhAAEd6AeJs8QY3wjkr4M+8EqQ== 2023-01-25T12:30:31.031 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.4] 2023-01-25T12:30:31.031 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDaINFjcqJRIRAAH/e0yxZiVqzf0jIupjWvLA== 2023-01-25T12:30:31.031 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.5] 2023-01-25T12:30:31.032 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDdINFjrTBkHxAAFWL9Sa3e4O74RbYHkCbfVw== 2023-01-25T12:30:31.032 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.6] 2023-01-25T12:30:31.032 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgINFjIt6zHBAA9VreTWD43r9gC3vLYfXm/A== 2023-01-25T12:30:31.032 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.7] 2023-01-25T12:30:31.033 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDjINFjd8GGGhAA/17eEGBZW/jqIQtGe+GtnQ== 2023-01-25T12:30:31.033 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2023-01-25T12:30:31.033 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDIINFjLa81FRAA/pieGEFFcOBpt+nHrM4itg== 2023-01-25T12:30:31.033 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2023-01-25T12:30:31.033 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDIINFjn0ajGBAA8zoNGJsGvwon11IMMORMMQ== 2023-01-25T12:30:31.050 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2023-01-25T12:30:31.050 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDIINFjYPtmDhAA1jz+hQrZJV23miXXmBCHQg== 2023-01-25T12:30:31.050 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2023-01-25T12:30:31.050 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDLINFj4m67IhAAjy+R2Z3A3nZrF7LdgjESng== 2023-01-25T12:30:31.051 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2023-01-25T12:30:31.051 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDOINFjnBXvIBAAloncHGYjSJHgZLbPrEMMaA== 2023-01-25T12:30:31.051 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2023-01-25T12:30:31.051 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDRINFjPrqoHhAAhA1NvjF8PsKR7qeRwskB6Q== 2023-01-25T12:30:31.051 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2023-01-25T12:30:31.052 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDUINFjbXg1HRAAoFTJYnp+vLxj1mDre9tpjQ== 2023-01-25T12:30:31.052 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2023-01-25T12:30:31.052 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDIINFjE3tLEhAAEd6AeJs8QY3wjkr4M+8EqQ== 2023-01-25T12:30:31.052 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2023-01-25T12:30:31.053 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDaINFjcqJRIRAAH/e0yxZiVqzf0jIupjWvLA== 2023-01-25T12:30:31.053 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2023-01-25T12:30:31.053 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDdINFjrTBkHxAAFWL9Sa3e4O74RbYHkCbfVw== 2023-01-25T12:30:31.053 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2023-01-25T12:30:31.053 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDgINFjIt6zHBAA9VreTWD43r9gC3vLYfXm/A== 2023-01-25T12:30:31.054 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2023-01-25T12:30:31.054 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDjINFjd8GGGhAA/17eEGBZW/jqIQtGe+GtnQ== 2023-01-25T12:30:31.054 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2023-01-25T12:30:31.054 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDIINFjLa81FRAA/pieGEFFcOBpt+nHrM4itg== 2023-01-25T12:30:31.055 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2023-01-25T12:30:31.055 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDIINFjn0ajGBAA8zoNGJsGvwon11IMMORMMQ== 2023-01-25T12:30:31.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T12:30:31.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T12:30:31.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.208 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.321 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T12:30:31.377 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T12:30:31.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.432 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.488 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.544 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T12:30:31.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T12:30:31.656 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T12:30:31.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T12:30:31.714 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T12:30:31.771 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T12:30:31.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T12:30:31.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T12:30:31.899 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T12:30:31.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T12:30:31.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T12:30:32.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T12:30:32.124 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T12:30:32.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T12:30:32.251 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T12:30:32.279 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:32.282 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T12:30:32.336 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T12:30:32.337 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T12:30:32.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T12:30:32.340 INFO:tasks.ceph.mon.a:Started 2023-01-25T12:30:32.341 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T12:30:32.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T12:30:32.343 INFO:tasks.ceph.mon.c:Started 2023-01-25T12:30:32.344 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T12:30:32.344 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T12:30:32.379 INFO:tasks.ceph.mon.b:Started 2023-01-25T12:30:32.379 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T12:30:32.380 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T12:30:32.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T12:30:32.383 INFO:tasks.ceph.mgr.y:Started 2023-01-25T12:30:32.383 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T12:30:32.383 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T12:30:32.386 INFO:tasks.ceph.mgr.x:Started 2023-01-25T12:30:32.386 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T12:30:32.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T12:30:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2023-01-25T12:30:39.039 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T12:30:39.040 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:39.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T12:30:39.072 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:39.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T12:30:39.139 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:39.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T12:30:39.206 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-01-25T12:30:39.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T12:30:39.274 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:39.275 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T12:30:39.301 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:39.302 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T12:30:39.368 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:39.369 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T12:30:39.435 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-01-25T12:30:39.436 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T12:30:39.502 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 8490bb7a-732b-4ae4-8be5-0da96b002c60 0 2023-01-25T12:30:39.760 INFO:teuthology.orchestra.run.smithi087.stdout:0 2023-01-25T12:30:39.773 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 7784e3c2-25af-4da0-9bac-93b4321855f4 1 2023-01-25T12:30:39.880 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:39.879+0000 7f72bbe9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T12:30:39.895 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:39.893+0000 7f2068feb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T12:30:39.953 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:39.951+0000 7f72bbe9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T12:30:39.968 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:39.966+0000 7f2068feb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T12:30:39.999 INFO:teuthology.orchestra.run.smithi087.stdout:1 2023-01-25T12:30:40.011 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 01ebcf61-a02d-47e9-aae9-19dbc0f88377 2 2023-01-25T12:30:40.044 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:40.042+0000 7f72bbe9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T12:30:40.063 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:40.061+0000 7f2068feb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T12:30:40.232 INFO:teuthology.orchestra.run.smithi087.stdout:2 2023-01-25T12:30:40.235 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:40.233+0000 7f72bbe9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T12:30:40.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new ea1c8296-924b-4fc4-a361-bb36d57ee193 3 2023-01-25T12:30:40.258 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:40.255+0000 7f2068feb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T12:30:40.462 INFO:teuthology.orchestra.run.smithi087.stdout:3 2023-01-25T12:30:40.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 5abcbc7a-90e4-4635-ac76-0dcbdb5b4678 4 2023-01-25T12:30:40.745 INFO:teuthology.orchestra.run.smithi087.stdout:4 2023-01-25T12:30:40.758 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new a7e08c97-ab32-4379-8ea2-4ab4aa171fa2 5 2023-01-25T12:30:41.013 INFO:teuthology.orchestra.run.smithi087.stdout:5 2023-01-25T12:30:41.025 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 3925ea4a-1e5a-4a48-9175-cf82680dc260 6 2023-01-25T12:30:41.058 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.056+0000 7f72bbe9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T12:30:41.083 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.081+0000 7f2068feb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T12:30:41.203 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.201+0000 7f72bbe9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T12:30:41.228 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.226+0000 7f2068feb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T12:30:41.250 INFO:teuthology.orchestra.run.smithi087.stdout:6 2023-01-25T12:30:41.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 2ddb564a-2de8-4b59-8c69-6d576d2a0544 7 2023-01-25T12:30:41.433 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.432+0000 7f72bbe9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T12:30:41.459 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.457+0000 7f2068feb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T12:30:41.486 INFO:teuthology.orchestra.run.smithi087.stdout:7 2023-01-25T12:30:41.498 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T12:30:41.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T12:30:41.501 INFO:tasks.ceph.osd.0:Started 2023-01-25T12:30:41.502 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T12:30:41.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T12:30:41.504 INFO:tasks.ceph.osd.1:Started 2023-01-25T12:30:41.504 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T12:30:41.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T12:30:41.507 INFO:tasks.ceph.osd.2:Started 2023-01-25T12:30:41.507 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T12:30:41.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T12:30:41.508 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.506+0000 7f72bbe9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T12:30:41.510 INFO:tasks.ceph.osd.3:Started 2023-01-25T12:30:41.511 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T12:30:41.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T12:30:41.513 INFO:tasks.ceph.osd.4:Started 2023-01-25T12:30:41.513 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T12:30:41.514 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T12:30:41.516 INFO:tasks.ceph.osd.5:Started 2023-01-25T12:30:41.516 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T12:30:41.516 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T12:30:41.518 INFO:tasks.ceph.osd.6:Started 2023-01-25T12:30:41.519 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T12:30:41.519 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T12:30:41.522 INFO:tasks.ceph.osd.7:Started 2023-01-25T12:30:41.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T12:30:41.540 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.538+0000 7f2068feb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T12:30:41.708 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.706+0000 7f2068feb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T12:30:41.709 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.708+0000 7f72bbe9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T12:30:41.782 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:41.782 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"fsid":"b623256f-e4e6-420d-b240-c3bfae6a45f5","created":"2023-01-25T12:30:38.682334+0000","modified":"2023-01-25T12:30:41.482467+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T12:30:41.482467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"8490bb7a-732b-4ae4-8be5-0da96b002c60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7784e3c2-25af-4da0-9bac-93b4321855f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"01ebcf61-a02d-47e9-aae9-19dbc0f88377","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ea1c8296-924b-4fc4-a361-bb36d57ee193","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5abcbc7a-90e4-4635-ac76-0dcbdb5b4678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e08c97-ab32-4379-8ea2-4ab4aa171fa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3925ea4a-1e5a-4a48-9175-cf82680dc260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2ddb564a-2de8-4b59-8c69-6d576d2a0544","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T12:30:41.795 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T12:30:41.796 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T12:30:41.972 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:41.970+0000 7f2068feb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T12:30:41.988 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:41.987+0000 7f72bbe9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T12:30:42.118 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-01-25T12:30:42.116+0000 7f51c9388580 -1 Falling back to public interface 2023-01-25T12:30:42.122 INFO:tasks.ceph.osd.4.smithi087.stderr:2023-01-25T12:30:42.120+0000 7f5e7537d580 -1 Falling back to public interface 2023-01-25T12:30:42.126 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-01-25T12:30:42.124+0000 7fad8cfc0580 -1 Falling back to public interface 2023-01-25T12:30:42.126 INFO:tasks.ceph.osd.5.smithi087.stderr:2023-01-25T12:30:42.124+0000 7fc3eef42580 -1 Falling back to public interface 2023-01-25T12:30:42.136 INFO:tasks.ceph.osd.7.smithi087.stderr:2023-01-25T12:30:42.134+0000 7f727d84d580 -1 Falling back to public interface 2023-01-25T12:30:42.145 INFO:tasks.ceph.osd.6.smithi087.stderr:2023-01-25T12:30:42.143+0000 7ff13e733580 -1 Falling back to public interface 2023-01-25T12:30:42.147 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-01-25T12:30:42.145+0000 7fe104dee580 -1 Falling back to public interface 2023-01-25T12:30:42.147 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-01-25T12:30:42.146+0000 7fd969732580 -1 Falling back to public interface 2023-01-25T12:30:42.156 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.155+0000 7f2068feb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T12:30:42.180 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.178+0000 7f72bbe9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T12:30:42.296 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.294+0000 7f2068feb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T12:30:42.323 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.322+0000 7f72bbe9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T12:30:42.410 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.408+0000 7f2068feb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T12:30:42.443 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.442+0000 7f72bbe9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T12:30:42.551 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.549+0000 7f2068feb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T12:30:42.586 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.585+0000 7f72bbe9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T12:30:42.602 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T12:30:42.629 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.627+0000 7f2068feb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T12:30:42.669 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.667+0000 7f72bbe9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T12:30:42.806 INFO:teuthology.misc.health.smithi067.stdout: 2023-01-25T12:30:42.807 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":10,"fsid":"b623256f-e4e6-420d-b240-c3bfae6a45f5","created":"2023-01-25T12:30:38.682334+0000","modified":"2023-01-25T12:30:41.482467+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T12:30:41.482467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"8490bb7a-732b-4ae4-8be5-0da96b002c60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7784e3c2-25af-4da0-9bac-93b4321855f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"01ebcf61-a02d-47e9-aae9-19dbc0f88377","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ea1c8296-924b-4fc4-a361-bb36d57ee193","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5abcbc7a-90e4-4635-ac76-0dcbdb5b4678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e08c97-ab32-4379-8ea2-4ab4aa171fa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3925ea4a-1e5a-4a48-9175-cf82680dc260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2ddb564a-2de8-4b59-8c69-6d576d2a0544","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T12:30:42.818 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T12:30:42.853 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.851+0000 7f2068feb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T12:30:42.899 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.898+0000 7f72bbe9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T12:30:42.931 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:42.929+0000 7f2068feb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T12:30:42.979 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:42.977+0000 7f72bbe9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T12:30:43.544 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:43.542+0000 7f2068feb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T12:30:43.620 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:43.618+0000 7f72bbe9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T12:30:43.796 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:43.794+0000 7f2068feb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T12:30:43.869 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:43.867+0000 7f2068feb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T12:30:43.882 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:43.880+0000 7f72bbe9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T12:30:43.949 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:43.948+0000 7f2068feb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T12:30:43.959 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:43.958+0000 7f72bbe9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T12:30:44.044 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:44.042+0000 7f72bbe9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T12:30:44.131 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:44.129+0000 7f2068feb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T12:30:44.202 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:44.200+0000 7f2068feb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T12:30:44.233 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:44.231+0000 7f72bbe9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T12:30:44.307 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:44.305+0000 7f72bbe9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T12:30:44.354 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:44.352+0000 7f2068feb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T12:30:44.464 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-01-25T12:30:44.463+0000 7f72bbe9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T12:30:45.384 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:45.382+0000 7f2022b64700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T12:30:45.386 INFO:tasks.ceph.mgr.x.smithi087.stderr:2023-01-25T12:30:45.384+0000 7f2022b64700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T12:30:45.629 INFO:tasks.ceph.osd.5.smithi087.stderr:2023-01-25T12:30:45.627+0000 7fc3eef42580 -1 osd.5 0 log_to_monitors true 2023-01-25T12:30:45.639 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-01-25T12:30:45.637+0000 7fad8cfc0580 -1 osd.3 0 log_to_monitors true 2023-01-25T12:30:45.639 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-01-25T12:30:45.638+0000 7f51c9388580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:30:45.642 INFO:tasks.ceph.osd.4.smithi087.stderr:2023-01-25T12:30:45.640+0000 7f5e7537d580 -1 osd.4 0 log_to_monitors true 2023-01-25T12:30:45.660 INFO:tasks.ceph.osd.7.smithi087.stderr:2023-01-25T12:30:45.659+0000 7f727d84d580 -1 osd.7 0 log_to_monitors true 2023-01-25T12:30:45.672 INFO:tasks.ceph.osd.6.smithi087.stderr:2023-01-25T12:30:45.670+0000 7ff13e733580 -1 osd.6 0 log_to_monitors true 2023-01-25T12:30:45.677 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-01-25T12:30:45.676+0000 7fd969732580 -1 osd.1 0 log_to_monitors true 2023-01-25T12:30:45.677 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-01-25T12:30:45.676+0000 7fe104dee580 -1 osd.2 0 log_to_monitors true 2023-01-25T12:30:47.394 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-01-25T12:30:47.392+0000 7f51bd730700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T12:30:47.401 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-01-25T12:30:47.399+0000 7fd95dada700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T12:30:47.401 INFO:tasks.ceph.osd.5.smithi087.stderr:2023-01-25T12:30:47.399+0000 7fc3e32ea700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T12:30:47.401 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-01-25T12:30:47.399+0000 7fe0f9196700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T12:30:47.402 INFO:tasks.ceph.osd.7.smithi087.stderr:2023-01-25T12:30:47.399+0000 7f7271bf5700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T12:30:47.402 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-01-25T12:30:47.399+0000 7fad81368700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T12:30:47.402 INFO:tasks.ceph.osd.4.smithi087.stderr:2023-01-25T12:30:47.399+0000 7f5e69725700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T12:30:47.403 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-01-25T12:30:47.402+0000 7f51b630c700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.406 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-01-25T12:30:47.405+0000 7fd9566b6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.407 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-01-25T12:30:47.406+0000 7fad79f44700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.408 INFO:tasks.ceph.osd.4.smithi087.stderr:2023-01-25T12:30:47.406+0000 7f5e62301700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.408 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-01-25T12:30:47.406+0000 7fe0f1d72700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.408 INFO:tasks.ceph.osd.5.smithi087.stderr:2023-01-25T12:30:47.406+0000 7fc3dbec6700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.409 INFO:tasks.ceph.osd.7.smithi087.stderr:2023-01-25T12:30:47.406+0000 7f726a7d1700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:47.583 INFO:tasks.ceph.osd.6.smithi087.stderr:2023-01-25T12:30:47.581+0000 7ff132adb700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T12:30:47.586 INFO:tasks.ceph.osd.6.smithi087.stderr:2023-01-25T12:30:47.584+0000 7ff12b6b7700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T12:30:49.624 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T12:30:49.963 INFO:teuthology.misc.health.smithi067.stdout: 2023-01-25T12:30:49.964 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":14,"fsid":"b623256f-e4e6-420d-b240-c3bfae6a45f5","created":"2023-01-25T12:30:38.682334+0000","modified":"2023-01-25T12:30:49.403009+0000","last_up_change":"2023-01-25T12:30:48.392287+0000","last_in_change":"2023-01-25T12:30:41.482467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T12:30:48.642758+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"8490bb7a-732b-4ae4-8be5-0da96b002c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":106354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6801","nonce":106354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6803","nonce":106354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":106354}]},"public_addr":"172.21.15.67:6800/106354","cluster_addr":"172.21.15.67:6801/106354","heartbeat_back_addr":"172.21.15.67:6803/106354","heartbeat_front_addr":"172.21.15.67:6802/106354","state":["exists","up"]},{"osd":1,"uuid":"7784e3c2-25af-4da0-9bac-93b4321855f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":106357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6813","nonce":106357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":106357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":106357}]},"public_addr":"172.21.15.67:6811/106357","cluster_addr":"172.21.15.67:6813/106357","heartbeat_back_addr":"172.21.15.67:6815/106357","heartbeat_front_addr":"172.21.15.67:6814/106357","state":["exists","up"]},{"osd":2,"uuid":"01ebcf61-a02d-47e9-aae9-19dbc0f88377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":106358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":106358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":106358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":106358}]},"public_addr":"172.21.15.67:6808/106358","cluster_addr":"172.21.15.67:6809/106358","heartbeat_back_addr":"172.21.15.67:6812/106358","heartbeat_front_addr":"172.21.15.67:6810/106358","state":["exists","up"]},{"osd":3,"uuid":"ea1c8296-924b-4fc4-a361-bb36d57ee193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":106355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":106355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6807","nonce":106355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":106355}]},"public_addr":"172.21.15.67:6804/106355","cluster_addr":"172.21.15.67:6805/106355","heartbeat_back_addr":"172.21.15.67:6807/106355","heartbeat_front_addr":"172.21.15.67:6806/106355","state":["exists","up"]},{"osd":4,"uuid":"5abcbc7a-90e4-4635-ac76-0dcbdb5b4678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":106242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6801","nonce":106242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":106242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":106242}]},"public_addr":"172.21.15.87:6800/106242","cluster_addr":"172.21.15.87:6801/106242","heartbeat_back_addr":"172.21.15.87:6803/106242","heartbeat_front_addr":"172.21.15.87:6802/106242","state":["exists","up"]},{"osd":5,"uuid":"a7e08c97-ab32-4379-8ea2-4ab4aa171fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":106243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6805","nonce":106243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":106243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":106243}]},"public_addr":"172.21.15.87:6804/106243","cluster_addr":"172.21.15.87:6805/106243","heartbeat_back_addr":"172.21.15.87:6807/106243","heartbeat_front_addr":"172.21.15.87:6806/106243","state":["exists","up"]},{"osd":6,"uuid":"3925ea4a-1e5a-4a48-9175-cf82680dc260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":106265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":106265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6815","nonce":106265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":106265}]},"public_addr":"172.21.15.87:6812/106265","cluster_addr":"172.21.15.87:6813/106265","heartbeat_back_addr":"172.21.15.87:6815/106265","heartbeat_front_addr":"172.21.15.87:6814/106265","state":["exists","up"]},{"osd":7,"uuid":"2ddb564a-2de8-4b59-8c69-6d576d2a0544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":106244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6809","nonce":106244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":106244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":106244}]},"public_addr":"172.21.15.87:6808/106244","cluster_addr":"172.21.15.87:6809/106244","heartbeat_back_addr":"172.21.15.87:6811/106244","heartbeat_front_addr":"172.21.15.87:6810/106244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.655782+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.674870+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.682823+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.613531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.622513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.644481+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.710854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T12:30:49.977 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T12:30:49.977 INFO:tasks.ceph:Creating RBD pool 2023-01-25T12:30:49.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T12:30:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'rbd' created 2023-01-25T12:30:50.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T12:30:51.426 INFO:teuthology.orchestra.run.smithi067.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T12:30:51.441 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T12:30:51.442 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T12:30:51.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T12:30:51.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T12:30:51.444 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T12:30:51.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:51.849 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":4,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":105579}]},"active_addr":"172.21.15.87:6816/105579","active_change":"2023-01-25T12:30:44.368058+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":707495102}]},{"addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":146798312}]},{"addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3780140911}]}]}} 2023-01-25T12:30:51.853 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T12:30:51.853 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T12:30:51.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T12:30:52.195 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"fsid":"b623256f-e4e6-420d-b240-c3bfae6a45f5","created":"2023-01-25T12:30:38.682334+0000","modified":"2023-01-25T12:30:51.414854+0000","last_up_change":"2023-01-25T12:30:48.392287+0000","last_in_change":"2023-01-25T12:30:41.482467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T12:30:48.642758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T12:30:50.340387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8490bb7a-732b-4ae4-8be5-0da96b002c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":106354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6801","nonce":106354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6803","nonce":106354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":106354}]},"public_addr":"172.21.15.67:6800/106354","cluster_addr":"172.21.15.67:6801/106354","heartbeat_back_addr":"172.21.15.67:6803/106354","heartbeat_front_addr":"172.21.15.67:6802/106354","state":["exists","up"]},{"osd":1,"uuid":"7784e3c2-25af-4da0-9bac-93b4321855f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":106357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6813","nonce":106357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":106357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":106357}]},"public_addr":"172.21.15.67:6811/106357","cluster_addr":"172.21.15.67:6813/106357","heartbeat_back_addr":"172.21.15.67:6815/106357","heartbeat_front_addr":"172.21.15.67:6814/106357","state":["exists","up"]},{"osd":2,"uuid":"01ebcf61-a02d-47e9-aae9-19dbc0f88377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":106358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":106358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":106358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":106358}]},"public_addr":"172.21.15.67:6808/106358","cluster_addr":"172.21.15.67:6809/106358","heartbeat_back_addr":"172.21.15.67:6812/106358","heartbeat_front_addr":"172.21.15.67:6810/106358","state":["exists","up"]},{"osd":3,"uuid":"ea1c8296-924b-4fc4-a361-bb36d57ee193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":106355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":106355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6807","nonce":106355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":106355}]},"public_addr":"172.21.15.67:6804/106355","cluster_addr":"172.21.15.67:6805/106355","heartbeat_back_addr":"172.21.15.67:6807/106355","heartbeat_front_addr":"172.21.15.67:6806/106355","state":["exists","up"]},{"osd":4,"uuid":"5abcbc7a-90e4-4635-ac76-0dcbdb5b4678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":106242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6801","nonce":106242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":106242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":106242}]},"public_addr":"172.21.15.87:6800/106242","cluster_addr":"172.21.15.87:6801/106242","heartbeat_back_addr":"172.21.15.87:6803/106242","heartbeat_front_addr":"172.21.15.87:6802/106242","state":["exists","up"]},{"osd":5,"uuid":"a7e08c97-ab32-4379-8ea2-4ab4aa171fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":106243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6805","nonce":106243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":106243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":106243}]},"public_addr":"172.21.15.87:6804/106243","cluster_addr":"172.21.15.87:6805/106243","heartbeat_back_addr":"172.21.15.87:6807/106243","heartbeat_front_addr":"172.21.15.87:6806/106243","state":["exists","up"]},{"osd":6,"uuid":"3925ea4a-1e5a-4a48-9175-cf82680dc260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":106265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":106265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6815","nonce":106265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":106265}]},"public_addr":"172.21.15.87:6812/106265","cluster_addr":"172.21.15.87:6813/106265","heartbeat_back_addr":"172.21.15.87:6815/106265","heartbeat_front_addr":"172.21.15.87:6814/106265","state":["exists","up"]},{"osd":7,"uuid":"2ddb564a-2de8-4b59-8c69-6d576d2a0544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":106244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6809","nonce":106244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":106244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":106244}]},"public_addr":"172.21.15.87:6808/106244","cluster_addr":"172.21.15.87:6809/106244","heartbeat_back_addr":"172.21.15.87:6811/106244","heartbeat_front_addr":"172.21.15.87:6810/106244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.655782+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.674870+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.682823+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.613531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.622513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.644481+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.710854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T12:30:52.211 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T12:30:52.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T12:30:52.550 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:52.550 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"fsid":"b623256f-e4e6-420d-b240-c3bfae6a45f5","created":"2023-01-25T12:30:38.682334+0000","modified":"2023-01-25T12:30:52.424264+0000","last_up_change":"2023-01-25T12:30:48.392287+0000","last_in_change":"2023-01-25T12:30:41.482467+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T12:30:48.642758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T12:30:50.340387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8490bb7a-732b-4ae4-8be5-0da96b002c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":106354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6801","nonce":106354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6803","nonce":106354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":106354}]},"public_addr":"172.21.15.67:6800/106354","cluster_addr":"172.21.15.67:6801/106354","heartbeat_back_addr":"172.21.15.67:6803/106354","heartbeat_front_addr":"172.21.15.67:6802/106354","state":["exists","up"]},{"osd":1,"uuid":"7784e3c2-25af-4da0-9bac-93b4321855f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":106357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6813","nonce":106357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":106357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":106357}]},"public_addr":"172.21.15.67:6811/106357","cluster_addr":"172.21.15.67:6813/106357","heartbeat_back_addr":"172.21.15.67:6815/106357","heartbeat_front_addr":"172.21.15.67:6814/106357","state":["exists","up"]},{"osd":2,"uuid":"01ebcf61-a02d-47e9-aae9-19dbc0f88377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":106358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":106358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":106358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":106358}]},"public_addr":"172.21.15.67:6808/106358","cluster_addr":"172.21.15.67:6809/106358","heartbeat_back_addr":"172.21.15.67:6812/106358","heartbeat_front_addr":"172.21.15.67:6810/106358","state":["exists","up"]},{"osd":3,"uuid":"ea1c8296-924b-4fc4-a361-bb36d57ee193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":106355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":106355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6807","nonce":106355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":106355}]},"public_addr":"172.21.15.67:6804/106355","cluster_addr":"172.21.15.67:6805/106355","heartbeat_back_addr":"172.21.15.67:6807/106355","heartbeat_front_addr":"172.21.15.67:6806/106355","state":["exists","up"]},{"osd":4,"uuid":"5abcbc7a-90e4-4635-ac76-0dcbdb5b4678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":106242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6801","nonce":106242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6803","nonce":106242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":106242}]},"public_addr":"172.21.15.87:6800/106242","cluster_addr":"172.21.15.87:6801/106242","heartbeat_back_addr":"172.21.15.87:6803/106242","heartbeat_front_addr":"172.21.15.87:6802/106242","state":["exists","up"]},{"osd":5,"uuid":"a7e08c97-ab32-4379-8ea2-4ab4aa171fa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":106243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6805","nonce":106243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6807","nonce":106243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":106243}]},"public_addr":"172.21.15.87:6804/106243","cluster_addr":"172.21.15.87:6805/106243","heartbeat_back_addr":"172.21.15.87:6807/106243","heartbeat_front_addr":"172.21.15.87:6806/106243","state":["exists","up"]},{"osd":6,"uuid":"3925ea4a-1e5a-4a48-9175-cf82680dc260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":106265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6813","nonce":106265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6815","nonce":106265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":106265}]},"public_addr":"172.21.15.87:6812/106265","cluster_addr":"172.21.15.87:6813/106265","heartbeat_back_addr":"172.21.15.87:6815/106265","heartbeat_front_addr":"172.21.15.87:6814/106265","state":["exists","up"]},{"osd":7,"uuid":"2ddb564a-2de8-4b59-8c69-6d576d2a0544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":106244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6809","nonce":106244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6811","nonce":106244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":106244}]},"public_addr":"172.21.15.87:6808/106244","cluster_addr":"172.21.15.87:6809/106244","heartbeat_back_addr":"172.21.15.87:6811/106244","heartbeat_front_addr":"172.21.15.87:6810/106244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.655782+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.674870+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.682823+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.613531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.622513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.644481+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T12:30:46.710854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T12:30:52.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T12:30:52.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T12:30:52.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T12:30:52.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T12:30:52.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T12:30:52.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T12:30:52.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T12:30:52.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T12:30:52.874 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:52.874 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:52.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T12:30:52.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T12:30:52.877 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:52.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T12:30:52.879 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:52.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T12:30:52.884 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:52.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T12:30:52.895 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:52.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T12:30:52.903 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:52.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T12:30:52.908 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:52.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T12:30:53.493 INFO:teuthology.orchestra.run.smithi067.stdout:55834574849 2023-01-25T12:30:53.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.5 2023-01-25T12:30:53.569 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:55834574849 2023-01-25T12:30:53.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T12:30:53.604 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:53.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-01-25T12:30:53.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-25T12:30:53.635 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:53.636 INFO:teuthology.orchestra.run.smithi067.stdout:55834574849 2023-01-25T12:30:53.644 INFO:teuthology.orchestra.run.smithi067.stdout:55834574849 2023-01-25T12:30:53.651 INFO:teuthology.orchestra.run.smithi067.stdout:55834574850 2023-01-25T12:30:53.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2023-01-25T12:30:53.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-25T12:30:53.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-01-25T12:30:53.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-25T12:30:54.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T12:30:54.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T12:30:54.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T12:30:54.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T12:30:54.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T12:30:54.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T12:30:54.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T12:30:54.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T12:30:55.026 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.5 2023-01-25T12:30:55.048 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.194 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.208 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.0 2023-01-25T12:30:55.209 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-25T12:30:55.227 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.265 INFO:teuthology.orchestra.run.smithi067.stdout:55834574852 2023-01-25T12:30:55.273 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.7 2023-01-25T12:30:55.281 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.2 2023-01-25T12:30:55.294 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.294 INFO:teuthology.orchestra.run.smithi067.stdout:55834574852 2023-01-25T12:30:55.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2023-01-25T12:30:55.310 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.311 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.1 2023-01-25T12:30:55.326 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.330 INFO:teuthology.orchestra.run.smithi067.stdout:55834574851 2023-01-25T12:30:55.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2023-01-25T12:30:55.344 DEBUG:teuthology.parallel:result is None 2023-01-25T12:30:55.345 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T12:30:55.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T12:30:55.717 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:55.720 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-01-25T12:30:55.731 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T12:30:54.374336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43224,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754931496,"statfs":{"total":773094113280,"available":773049851904,"internally_reserved":0,"allocated":2592768,"data_stored":1771984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543826+0000","last_change":"2023-01-25T12:30:51.428218+0000","last_active":"2023-01-25T12:30:52.543826+0000","last_peered":"2023-01-25T12:30:52.543826+0000","last_clean":"2023-01-25T12:30:52.543826+0000","last_became_active":"2023-01-25T12:30:51.427901+0000","last_became_peered":"2023-01-25T12:30:51.427901+0000","last_unstale":"2023-01-25T12:30:52.543826+0000","last_undegraded":"2023-01-25T12:30:52.543826+0000","last_fullsized":"2023-01-25T12:30:52.543826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.114928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543738+0000","last_change":"2023-01-25T12:30:51.631017+0000","last_active":"2023-01-25T12:30:52.543738+0000","last_peered":"2023-01-25T12:30:52.543738+0000","last_clean":"2023-01-25T12:30:52.543738+0000","last_became_active":"2023-01-25T12:30:51.630772+0000","last_became_peered":"2023-01-25T12:30:51.630772+0000","last_unstale":"2023-01-25T12:30:52.543738+0000","last_undegraded":"2023-01-25T12:30:52.543738+0000","last_fullsized":"2023-01-25T12:30:52.543738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.544012+0000","last_change":"2023-01-25T12:30:51.632143+0000","last_active":"2023-01-25T12:30:52.544012+0000","last_peered":"2023-01-25T12:30:52.544012+0000","last_clean":"2023-01-25T12:30:52.544012+0000","last_became_active":"2023-01-25T12:30:51.631736+0000","last_became_peered":"2023-01-25T12:30:51.631736+0000","last_unstale":"2023-01-25T12:30:52.544012+0000","last_undegraded":"2023-01-25T12:30:52.544012+0000","last_fullsized":"2023-01-25T12:30:52.544012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.543113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543952+0000","last_change":"2023-01-25T12:30:51.426346+0000","last_active":"2023-01-25T12:30:52.543952+0000","last_peered":"2023-01-25T12:30:52.543952+0000","last_clean":"2023-01-25T12:30:52.543952+0000","last_became_active":"2023-01-25T12:30:51.426037+0000","last_became_peered":"2023-01-25T12:30:51.426037+0000","last_unstale":"2023-01-25T12:30:52.543952+0000","last_undegraded":"2023-01-25T12:30:52.543952+0000","last_fullsized":"2023-01-25T12:30:52.543952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:53.593718+0000","last_change":"2023-01-25T12:30:50.423991+0000","last_active":"2023-01-25T12:30:53.593718+0000","last_peered":"2023-01-25T12:30:53.593718+0000","last_clean":"2023-01-25T12:30:53.593718+0000","last_became_active":"2023-01-25T12:30:50.423619+0000","last_became_peered":"2023-01-25T12:30:50.423619+0000","last_unstale":"2023-01-25T12:30:53.593718+0000","last_undegraded":"2023-01-25T12:30:53.593718+0000","last_fullsized":"2023-01-25T12:30:53.593718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_clean_scrub_stamp":"2023-01-25T12:30:49.403009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:18.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.428250+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.427641+0000","last_became_peered":"2023-01-25T12:30:51.427641+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.028536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543453+0000","last_change":"2023-01-25T12:30:51.757954+0000","last_active":"2023-01-25T12:30:52.543453+0000","last_peered":"2023-01-25T12:30:52.543453+0000","last_clean":"2023-01-25T12:30:52.543453+0000","last_became_active":"2023-01-25T12:30:51.757636+0000","last_became_peered":"2023-01-25T12:30:51.757636+0000","last_unstale":"2023-01-25T12:30:52.543453+0000","last_undegraded":"2023-01-25T12:30:52.543453+0000","last_fullsized":"2023-01-25T12:30:52.543453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.702504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.427271+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.426937+0000","last_became_peered":"2023-01-25T12:30:51.426937+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:55.311470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543693+0000","last_change":"2023-01-25T12:30:51.427849+0000","last_active":"2023-01-25T12:30:52.543693+0000","last_peered":"2023-01-25T12:30:52.543693+0000","last_clean":"2023-01-25T12:30:52.543693+0000","last_became_active":"2023-01-25T12:30:51.427408+0000","last_became_peered":"2023-01-25T12:30:51.427408+0000","last_unstale":"2023-01-25T12:30:52.543693+0000","last_undegraded":"2023-01-25T12:30:52.543693+0000","last_fullsized":"2023-01-25T12:30:52.543693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:50.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.456}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4420000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3890000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":954.18200000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T12:30:55.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T12:30:56.064 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:56.066 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-01-25T12:30:56.077 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T12:30:54.374336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43224,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754931496,"statfs":{"total":773094113280,"available":773049851904,"internally_reserved":0,"allocated":2592768,"data_stored":1771984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543826+0000","last_change":"2023-01-25T12:30:51.428218+0000","last_active":"2023-01-25T12:30:52.543826+0000","last_peered":"2023-01-25T12:30:52.543826+0000","last_clean":"2023-01-25T12:30:52.543826+0000","last_became_active":"2023-01-25T12:30:51.427901+0000","last_became_peered":"2023-01-25T12:30:51.427901+0000","last_unstale":"2023-01-25T12:30:52.543826+0000","last_undegraded":"2023-01-25T12:30:52.543826+0000","last_fullsized":"2023-01-25T12:30:52.543826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.114928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543738+0000","last_change":"2023-01-25T12:30:51.631017+0000","last_active":"2023-01-25T12:30:52.543738+0000","last_peered":"2023-01-25T12:30:52.543738+0000","last_clean":"2023-01-25T12:30:52.543738+0000","last_became_active":"2023-01-25T12:30:51.630772+0000","last_became_peered":"2023-01-25T12:30:51.630772+0000","last_unstale":"2023-01-25T12:30:52.543738+0000","last_undegraded":"2023-01-25T12:30:52.543738+0000","last_fullsized":"2023-01-25T12:30:52.543738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.544012+0000","last_change":"2023-01-25T12:30:51.632143+0000","last_active":"2023-01-25T12:30:52.544012+0000","last_peered":"2023-01-25T12:30:52.544012+0000","last_clean":"2023-01-25T12:30:52.544012+0000","last_became_active":"2023-01-25T12:30:51.631736+0000","last_became_peered":"2023-01-25T12:30:51.631736+0000","last_unstale":"2023-01-25T12:30:52.544012+0000","last_undegraded":"2023-01-25T12:30:52.544012+0000","last_fullsized":"2023-01-25T12:30:52.544012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.543113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543952+0000","last_change":"2023-01-25T12:30:51.426346+0000","last_active":"2023-01-25T12:30:52.543952+0000","last_peered":"2023-01-25T12:30:52.543952+0000","last_clean":"2023-01-25T12:30:52.543952+0000","last_became_active":"2023-01-25T12:30:51.426037+0000","last_became_peered":"2023-01-25T12:30:51.426037+0000","last_unstale":"2023-01-25T12:30:52.543952+0000","last_undegraded":"2023-01-25T12:30:52.543952+0000","last_fullsized":"2023-01-25T12:30:52.543952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:53.593718+0000","last_change":"2023-01-25T12:30:50.423991+0000","last_active":"2023-01-25T12:30:53.593718+0000","last_peered":"2023-01-25T12:30:53.593718+0000","last_clean":"2023-01-25T12:30:53.593718+0000","last_became_active":"2023-01-25T12:30:50.423619+0000","last_became_peered":"2023-01-25T12:30:50.423619+0000","last_unstale":"2023-01-25T12:30:53.593718+0000","last_undegraded":"2023-01-25T12:30:53.593718+0000","last_fullsized":"2023-01-25T12:30:53.593718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_clean_scrub_stamp":"2023-01-25T12:30:49.403009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:18.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.428250+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.427641+0000","last_became_peered":"2023-01-25T12:30:51.427641+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.028536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543453+0000","last_change":"2023-01-25T12:30:51.757954+0000","last_active":"2023-01-25T12:30:52.543453+0000","last_peered":"2023-01-25T12:30:52.543453+0000","last_clean":"2023-01-25T12:30:52.543453+0000","last_became_active":"2023-01-25T12:30:51.757636+0000","last_became_peered":"2023-01-25T12:30:51.757636+0000","last_unstale":"2023-01-25T12:30:52.543453+0000","last_undegraded":"2023-01-25T12:30:52.543453+0000","last_fullsized":"2023-01-25T12:30:52.543453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.702504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.427271+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.426937+0000","last_became_peered":"2023-01-25T12:30:51.426937+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:55.311470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543693+0000","last_change":"2023-01-25T12:30:51.427849+0000","last_active":"2023-01-25T12:30:52.543693+0000","last_peered":"2023-01-25T12:30:52.543693+0000","last_clean":"2023-01-25T12:30:52.543693+0000","last_became_active":"2023-01-25T12:30:51.427408+0000","last_became_peered":"2023-01-25T12:30:51.427408+0000","last_unstale":"2023-01-25T12:30:52.543693+0000","last_undegraded":"2023-01-25T12:30:52.543693+0000","last_fullsized":"2023-01-25T12:30:52.543693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:50.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.456}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4420000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3890000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":954.18200000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T12:30:56.079 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T12:30:56.079 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T12:30:56.080 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T12:30:56.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T12:30:56.454 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:56.454 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T12:30:56.468 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T12:30:56.469 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-25T12:30:56.482 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T12:30:56.483 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri6-testing-2023-01-24-0746'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri6-testing-2023-01-24-0746', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T12:30:56.483 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-25T12:30:56.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-25T12:30:56.836 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-25T12:30:56.837 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T12:30:56.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T12:30:57.172 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-01-25T12:30:57.180 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T12:30:56.374897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43224,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754931496,"statfs":{"total":773094113280,"available":773049851904,"internally_reserved":0,"allocated":2592768,"data_stored":1771984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000561"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543826+0000","last_change":"2023-01-25T12:30:51.428218+0000","last_active":"2023-01-25T12:30:52.543826+0000","last_peered":"2023-01-25T12:30:52.543826+0000","last_clean":"2023-01-25T12:30:52.543826+0000","last_became_active":"2023-01-25T12:30:51.427901+0000","last_became_peered":"2023-01-25T12:30:51.427901+0000","last_unstale":"2023-01-25T12:30:52.543826+0000","last_undegraded":"2023-01-25T12:30:52.543826+0000","last_fullsized":"2023-01-25T12:30:52.543826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.114928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543738+0000","last_change":"2023-01-25T12:30:51.631017+0000","last_active":"2023-01-25T12:30:52.543738+0000","last_peered":"2023-01-25T12:30:52.543738+0000","last_clean":"2023-01-25T12:30:52.543738+0000","last_became_active":"2023-01-25T12:30:51.630772+0000","last_became_peered":"2023-01-25T12:30:51.630772+0000","last_unstale":"2023-01-25T12:30:52.543738+0000","last_undegraded":"2023-01-25T12:30:52.543738+0000","last_fullsized":"2023-01-25T12:30:52.543738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.544012+0000","last_change":"2023-01-25T12:30:51.632143+0000","last_active":"2023-01-25T12:30:52.544012+0000","last_peered":"2023-01-25T12:30:52.544012+0000","last_clean":"2023-01-25T12:30:52.544012+0000","last_became_active":"2023-01-25T12:30:51.631736+0000","last_became_peered":"2023-01-25T12:30:51.631736+0000","last_unstale":"2023-01-25T12:30:52.544012+0000","last_undegraded":"2023-01-25T12:30:52.544012+0000","last_fullsized":"2023-01-25T12:30:52.544012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.543113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543952+0000","last_change":"2023-01-25T12:30:51.426346+0000","last_active":"2023-01-25T12:30:52.543952+0000","last_peered":"2023-01-25T12:30:52.543952+0000","last_clean":"2023-01-25T12:30:52.543952+0000","last_became_active":"2023-01-25T12:30:51.426037+0000","last_became_peered":"2023-01-25T12:30:51.426037+0000","last_unstale":"2023-01-25T12:30:52.543952+0000","last_undegraded":"2023-01-25T12:30:52.543952+0000","last_fullsized":"2023-01-25T12:30:52.543952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:53.593718+0000","last_change":"2023-01-25T12:30:50.423991+0000","last_active":"2023-01-25T12:30:53.593718+0000","last_peered":"2023-01-25T12:30:53.593718+0000","last_clean":"2023-01-25T12:30:53.593718+0000","last_became_active":"2023-01-25T12:30:50.423619+0000","last_became_peered":"2023-01-25T12:30:50.423619+0000","last_unstale":"2023-01-25T12:30:53.593718+0000","last_undegraded":"2023-01-25T12:30:53.593718+0000","last_fullsized":"2023-01-25T12:30:53.593718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_clean_scrub_stamp":"2023-01-25T12:30:49.403009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:18.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.428250+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.427641+0000","last_became_peered":"2023-01-25T12:30:51.427641+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.028536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543453+0000","last_change":"2023-01-25T12:30:51.757954+0000","last_active":"2023-01-25T12:30:52.543453+0000","last_peered":"2023-01-25T12:30:52.543453+0000","last_clean":"2023-01-25T12:30:52.543453+0000","last_became_active":"2023-01-25T12:30:51.757636+0000","last_became_peered":"2023-01-25T12:30:51.757636+0000","last_unstale":"2023-01-25T12:30:52.543453+0000","last_undegraded":"2023-01-25T12:30:52.543453+0000","last_fullsized":"2023-01-25T12:30:52.543453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.702504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.427271+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.426937+0000","last_became_peered":"2023-01-25T12:30:51.426937+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:55.311470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543693+0000","last_change":"2023-01-25T12:30:51.427849+0000","last_active":"2023-01-25T12:30:52.543693+0000","last_peered":"2023-01-25T12:30:52.543693+0000","last_clean":"2023-01-25T12:30:52.543693+0000","last_became_active":"2023-01-25T12:30:51.427408+0000","last_became_peered":"2023-01-25T12:30:51.427408+0000","last_unstale":"2023-01-25T12:30:52.543693+0000","last_undegraded":"2023-01-25T12:30:52.543693+0000","last_fullsized":"2023-01-25T12:30:52.543693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:50.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.456}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4420000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3890000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":954.18200000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T12:30:57.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T12:30:57.514 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-01-25T12:30:57.516 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-01-25T12:30:57.527 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T12:30:56.374897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":43224,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754931496,"statfs":{"total":773094113280,"available":773049851904,"internally_reserved":0,"allocated":2592768,"data_stored":1771984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000561"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543826+0000","last_change":"2023-01-25T12:30:51.428218+0000","last_active":"2023-01-25T12:30:52.543826+0000","last_peered":"2023-01-25T12:30:52.543826+0000","last_clean":"2023-01-25T12:30:52.543826+0000","last_became_active":"2023-01-25T12:30:51.427901+0000","last_became_peered":"2023-01-25T12:30:51.427901+0000","last_unstale":"2023-01-25T12:30:52.543826+0000","last_undegraded":"2023-01-25T12:30:52.543826+0000","last_fullsized":"2023-01-25T12:30:52.543826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.114928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543738+0000","last_change":"2023-01-25T12:30:51.631017+0000","last_active":"2023-01-25T12:30:52.543738+0000","last_peered":"2023-01-25T12:30:52.543738+0000","last_clean":"2023-01-25T12:30:52.543738+0000","last_became_active":"2023-01-25T12:30:51.630772+0000","last_became_peered":"2023-01-25T12:30:51.630772+0000","last_unstale":"2023-01-25T12:30:52.543738+0000","last_undegraded":"2023-01-25T12:30:52.543738+0000","last_fullsized":"2023-01-25T12:30:52.543738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.999476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.544012+0000","last_change":"2023-01-25T12:30:51.632143+0000","last_active":"2023-01-25T12:30:52.544012+0000","last_peered":"2023-01-25T12:30:52.544012+0000","last_clean":"2023-01-25T12:30:52.544012+0000","last_became_active":"2023-01-25T12:30:51.631736+0000","last_became_peered":"2023-01-25T12:30:51.631736+0000","last_unstale":"2023-01-25T12:30:52.544012+0000","last_undegraded":"2023-01-25T12:30:52.544012+0000","last_fullsized":"2023-01-25T12:30:52.544012+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:01.543113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543952+0000","last_change":"2023-01-25T12:30:51.426346+0000","last_active":"2023-01-25T12:30:52.543952+0000","last_peered":"2023-01-25T12:30:52.543952+0000","last_clean":"2023-01-25T12:30:52.543952+0000","last_became_active":"2023-01-25T12:30:51.426037+0000","last_became_peered":"2023-01-25T12:30:51.426037+0000","last_unstale":"2023-01-25T12:30:52.543952+0000","last_undegraded":"2023-01-25T12:30:52.543952+0000","last_fullsized":"2023-01-25T12:30:52.543952+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:53.593718+0000","last_change":"2023-01-25T12:30:50.423991+0000","last_active":"2023-01-25T12:30:53.593718+0000","last_peered":"2023-01-25T12:30:53.593718+0000","last_clean":"2023-01-25T12:30:53.593718+0000","last_became_active":"2023-01-25T12:30:50.423619+0000","last_became_peered":"2023-01-25T12:30:50.423619+0000","last_unstale":"2023-01-25T12:30:53.593718+0000","last_undegraded":"2023-01-25T12:30:53.593718+0000","last_fullsized":"2023-01-25T12:30:53.593718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:49.403009+0000","last_clean_scrub_stamp":"2023-01-25T12:30:49.403009+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:18.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.428250+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.427641+0000","last_became_peered":"2023-01-25T12:30:51.427641+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:58.028536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543453+0000","last_change":"2023-01-25T12:30:51.757954+0000","last_active":"2023-01-25T12:30:52.543453+0000","last_peered":"2023-01-25T12:30:52.543453+0000","last_clean":"2023-01-25T12:30:52.543453+0000","last_became_active":"2023-01-25T12:30:51.757636+0000","last_became_peered":"2023-01-25T12:30:51.757636+0000","last_unstale":"2023-01-25T12:30:52.543453+0000","last_undegraded":"2023-01-25T12:30:52.543453+0000","last_fullsized":"2023-01-25T12:30:52.543453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:32:03.702504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543530+0000","last_change":"2023-01-25T12:30:51.427271+0000","last_active":"2023-01-25T12:30:52.543530+0000","last_peered":"2023-01-25T12:30:52.543530+0000","last_clean":"2023-01-25T12:30:52.543530+0000","last_became_active":"2023-01-25T12:30:51.426937+0000","last_became_peered":"2023-01-25T12:30:51.426937+0000","last_unstale":"2023-01-25T12:30:52.543530+0000","last_undegraded":"2023-01-25T12:30:52.543530+0000","last_fullsized":"2023-01-25T12:30:52.543530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:55.311470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T12:30:52.543693+0000","last_change":"2023-01-25T12:30:51.427849+0000","last_active":"2023-01-25T12:30:52.543693+0000","last_peered":"2023-01-25T12:30:52.543693+0000","last_clean":"2023-01-25T12:30:52.543693+0000","last_became_active":"2023-01-25T12:30:51.427408+0000","last_became_peered":"2023-01-25T12:30:51.427408+0000","last_unstale":"2023-01-25T12:30:52.543693+0000","last_undegraded":"2023-01-25T12:30:52.543693+0000","last_fullsized":"2023-01-25T12:30:52.543693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T12:30:50.411059+0000","last_clean_scrub_stamp":"2023-01-25T12:30:50.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:31:50.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.456}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4670000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5984,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365856,"statfs":{"total":96636764160,"available":96630636544,"internally_reserved":0,"allocated":618496,"data_stored":513678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4420000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3890000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":954.18200000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T12:30:57.529 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T12:30:57.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-25T12:30:57.862 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 17 2023-01-25T12:30:57.862 INFO:teuthology.orchestra.run.smithi067.stdout:fsid b623256f-e4e6-420d-b240-c3bfae6a45f5 2023-01-25T12:30:57.862 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-01-25T12:30:38.682334+0000 2023-01-25T12:30:57.862 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2023-01-25T12:30:52.424264+0000 2023-01-25T12:30:57.863 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-25T12:30:57.863 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 4 2023-01-25T12:30:57.863 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2023-01-25T12:30:57.863 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2023-01-25T12:30:57.864 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2023-01-25T12:30:57.864 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2023-01-25T12:30:57.864 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client jewel 2023-01-25T12:30:57.864 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release reef 2023-01-25T12:30:57.864 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2023-01-25T12:30:57.865 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-25T12:30:57.865 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-25T12:30:57.865 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 8 2023-01-25T12:30:57.865 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6800/106354 v2:172.21.15.67:6801/106354 exists,up 8490bb7a-732b-4ae4-8be5-0da96b002c60 2023-01-25T12:30:57.865 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6811/106357 v2:172.21.15.67:6813/106357 exists,up 7784e3c2-25af-4da0-9bac-93b4321855f4 2023-01-25T12:30:57.866 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6808/106358 v2:172.21.15.67:6809/106358 exists,up 01ebcf61-a02d-47e9-aae9-19dbc0f88377 2023-01-25T12:30:57.866 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6804/106355 v2:172.21.15.67:6805/106355 exists,up ea1c8296-924b-4fc4-a361-bb36d57ee193 2023-01-25T12:30:57.866 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6800/106242 v2:172.21.15.87:6801/106242 exists,up 5abcbc7a-90e4-4635-ac76-0dcbdb5b4678 2023-01-25T12:30:57.866 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6804/106243 v2:172.21.15.87:6805/106243 exists,up a7e08c97-ab32-4379-8ea2-4ab4aa171fa2 2023-01-25T12:30:57.866 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6812/106265 v2:172.21.15.87:6813/106265 exists,up 3925ea4a-1e5a-4a48-9175-cf82680dc260 2023-01-25T12:30:57.867 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6808/106244 v2:172.21.15.87:6809/106244 exists,up 2ddb564a-2de8-4b59-8c69-6d576d2a0544 2023-01-25T12:30:57.876 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6800/106354 v2:172.21.15.67:6801/106354 exists,up 8490bb7a-732b-4ae4-8be5-0da96b002c60', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6811/106357 v2:172.21.15.67:6813/106357 exists,up 7784e3c2-25af-4da0-9bac-93b4321855f4', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6808/106358 v2:172.21.15.67:6809/106358 exists,up 01ebcf61-a02d-47e9-aae9-19dbc0f88377', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.67:6804/106355 v2:172.21.15.67:6805/106355 exists,up ea1c8296-924b-4fc4-a361-bb36d57ee193', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6800/106242 v2:172.21.15.87:6801/106242 exists,up 5abcbc7a-90e4-4635-ac76-0dcbdb5b4678', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6804/106243 v2:172.21.15.87:6805/106243 exists,up a7e08c97-ab32-4379-8ea2-4ab4aa171fa2', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6812/106265 v2:172.21.15.87:6813/106265 exists,up 3925ea4a-1e5a-4a48-9175-cf82680dc260', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.87:6808/106244 v2:172.21.15.87:6809/106244 exists,up 2ddb564a-2de8-4b59-8c69-6d576d2a0544'] 2023-01-25T12:30:57.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-25T12:30:58.003 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-01-25T12:30:58.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mon.a", 2023-01-25T12:30:58.003 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster": "ceph", 2023-01-25T12:30:58.004 INFO:teuthology.orchestra.run.smithi067.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-25T12:30:58.004 INFO:teuthology.orchestra.run.smithi067.stdout: "admin_socket_mode": "", 2023-01-25T12:30:58.004 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-25T12:30:58.004 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_client_required": "cephx, none", 2023-01-25T12:30:58.004 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_cluster_required": "cephx", 2023-01-25T12:30:58.005 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_debug": "true", 2023-01-25T12:30:58.005 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-25T12:30:58.005 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-25T12:30:58.005 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_service_required": "cephx", 2023-01-25T12:30:58.005 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-25T12:30:58.006 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_supported": "cephx", 2023-01-25T12:30:58.006 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio": "true", 2023-01-25T12:30:58.006 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-25T12:30:58.006 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_poll_ms": "250", 2023-01-25T12:30:58.006 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_reap_max": "16", 2023-01-25T12:30:58.007 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_async_discard": "false", 2023-01-25T12:30:58.007 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_block_size": "4096", 2023-01-25T12:30:58.007 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio": "false", 2023-01-25T12:30:58.007 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-25T12:30:58.007 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-25T12:30:58.008 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_inflight_ios": "false", 2023-01-25T12:30:58.008 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_enable_discard": "false", 2023-01-25T12:30:58.008 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_flock_retry": "3", 2023-01-25T12:30:58.008 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-25T12:30:58.008 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_inject_crash": "0", 2023-01-25T12:30:58.009 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-25T12:30:58.009 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring": "false", 2023-01-25T12:30:58.009 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring_hipri": "false", 2023-01-25T12:30:58.009 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-25T12:30:58.009 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-25T12:30:58.010 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-25T12:30:58.010 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-25T12:30:58.010 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_type": "", 2023-01-25T12:30:58.010 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_alloc_size": "1048576", 2023-01-25T12:30:58.011 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_allocator": "hybrid", 2023-01-25T12:30:58.011 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_buffered_io": "true", 2023-01-25T12:30:58.011 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_for_zeros": "false", 2023-01-25T12:30:58.011 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-25T12:30:58.011 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-25T12:30:58.012 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_compact_log_sync": "false", 2023-01-25T12:30:58.012 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-25T12:30:58.012 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-25T12:30:58.012 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-25T12:30:58.012 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_max_log_runway": "4194304", 2023-01-25T12:30:58.013 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_max_prefetch": "1048576", 2023-01-25T12:30:58.013 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_min_flush_size": "524288", 2023-01-25T12:30:58.013 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_min_log_runway": "1048576", 2023-01-25T12:30:58.014 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_replay_recovery": "false", 2023-01-25T12:30:58.014 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-25T12:30:58.014 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-25T12:30:58.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_sync_write": "false", 2023-01-25T12:30:58.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-25T12:30:58.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-25T12:30:58.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-25T12:30:58.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_allocation_from_file": "true", 2023-01-25T12:30:58.016 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_allocator": "hybrid", 2023-01-25T12:30:58.016 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-25T12:30:58.016 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-25T12:30:58.016 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-25T12:30:58.017 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-25T12:30:58.017 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-25T12:30:58.017 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-25T12:30:58.017 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-25T12:30:58.017 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_create": "true", 2023-01-25T12:30:58.018 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_create": "false", 2023-01-25T12:30:58.018 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_path": "", 2023-01-25T12:30:58.018 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_size": "0", 2023-01-25T12:30:58.018 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_path": "", 2023-01-25T12:30:58.018 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_preallocate_file": "false", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_size": "107374182400", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_create": "false", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_path": "", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_size": "100663296", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs": "true", 2023-01-25T12:30:58.019 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-25T12:30:58.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-25T12:30:58.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-25T12:30:58.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-25T12:30:58.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T12:30:58.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T12:30:58.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-25T12:30:58.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T12:30:58.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_autotune": "true", 2023-01-25T12:30:58.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-25T12:30:58.022 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-25T12:30:58.022 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-25T12:30:58.022 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-25T12:30:58.022 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size": "0", 2023-01-25T12:30:58.023 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-25T12:30:58.023 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-25T12:30:58.023 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-25T12:30:58.023 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-25T12:30:58.023 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_type": "2q", 2023-01-25T12:30:58.024 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-25T12:30:58.024 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_clone_cow": "true", 2023-01-25T12:30:58.024 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-25T12:30:58.024 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-25T12:30:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-25T12:30:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-25T12:30:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-25T12:30:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-25T12:30:58.026 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-25T12:30:58.026 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_mode": "none", 2023-01-25T12:30:58.026 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-25T12:30:58.026 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_csum_type": "crc32c", 2023-01-25T12:30:58.027 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-25T12:30:58.027 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_freelist": "false", 2023-01-25T12:30:58.027 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-25T12:30:58.027 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-01-25T12:30:58.027 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-25T12:30:58.028 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-25T12:30:58.028 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-25T12:30:58.028 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-25T12:30:58.028 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-25T12:30:58.028 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-25T12:30:58.029 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-25T12:30:58.029 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-25T12:30:58.029 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-25T12:30:58.029 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-25T12:30:58.029 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-25T12:30:58.030 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-25T12:30:58.030 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_small_allocations": "0", 2023-01-25T12:30:58.030 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-25T12:30:58.030 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_default_buffered_read": "true", 2023-01-25T12:30:58.030 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_default_buffered_write": "false", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-25T12:30:58.031 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-25T12:30:58.032 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-25T12:30:58.032 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-25T12:30:58.032 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fail_eio": "false", 2023-01-25T12:30:58.032 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-25T12:30:58.033 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-25T12:30:58.033 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-25T12:30:58.033 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-25T12:30:58.033 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-25T12:30:58.033 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-25T12:30:58.034 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mount": "false", 2023-01-25T12:30:58.034 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-25T12:30:58.034 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_umount": "false", 2023-01-25T12:30:58.034 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-25T12:30:58.034 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-25T12:30:58.035 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-25T12:30:58.035 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-25T12:30:58.035 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-25T12:30:58.035 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-25T12:30:58.036 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-25T12:30:58.036 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-25T12:30:58.036 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_ignore_data_csum": "false", 2023-01-25T12:30:58.036 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-25T12:30:58.036 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-25T12:30:58.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-25T12:30:58.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-25T12:30:58.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_op_age": "5.000000", 2023-01-25T12:30:58.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_alloc_size": "0", 2023-01-25T12:30:58.037 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size": "0", 2023-01-25T12:30:58.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-25T12:30:58.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-25T12:30:58.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-25T12:30:58.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_deferred_txc": "32", 2023-01-25T12:30:58.038 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size": "0", 2023-01-25T12:30:58.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-25T12:30:58.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-25T12:30:58.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_nid_prealloc": "1024", 2023-01-25T12:30:58.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-25T12:30:58.039 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-25T12:30:58.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-25T12:30:58.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-25T12:30:58.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_retry_disk_reads": "3", 2023-01-25T12:30:58.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_cf": "true", 2023-01-25T12:30:58.040 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-25T12:30:58.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-01-25T12:30:58.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-25T12:30:58.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-25T12:30:58.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-25T12:30:58.041 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-25T12:30:58.042 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_mem": "512", 2023-01-25T12:30:58.042 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-25T12:30:58.042 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-25T12:30:58.042 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-25T12:30:58.042 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-25T12:30:58.043 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-25T12:30:58.043 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-25T12:30:58.043 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-25T12:30:58.043 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_tracing": "false", 2023-01-25T12:30:58.044 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-25T12:30:58.044 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-25T12:30:58.044 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-25T12:30:58.044 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-25T12:30:58.044 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-25T12:30:58.045 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-25T12:30:58.045 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-25T12:30:58.045 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-25T12:30:58.045 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-25T12:30:58.045 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_zero_block_detection": "false", 2023-01-25T12:30:58.046 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-25T12:30:58.046 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-25T12:30:58.046 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-25T12:30:58.046 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-25T12:30:58.046 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-25T12:30:58.047 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-25T12:30:58.047 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-25T12:30:58.047 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-25T12:30:58.047 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-25T12:30:58.047 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-25T12:30:58.048 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-25T12:30:58.048 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-25T12:30:58.048 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-25T12:30:58.048 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_cluster_require_signatures": "false", 2023-01-25T12:30:58.048 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_cluster_require_version": "2", 2023-01-25T12:30:58.049 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_require_signatures": "false", 2023-01-25T12:30:58.049 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_require_version": "2", 2023-01-25T12:30:58.049 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_service_require_signatures": "false", 2023-01-25T12:30:58.049 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_service_require_version": "2", 2023-01-25T12:30:58.050 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_sign_messages": "true", 2023-01-25T12:30:58.050 INFO:teuthology.orchestra.run.smithi067.stdout: "chdir": "", 2023-01-25T12:30:58.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client_acl_type": "", 2023-01-25T12:30:58.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client_asio_thread_count": "2", 2023-01-25T12:30:58.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client_cache_mid": "0.750000", 2023-01-25T12:30:58.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_cache_size": "16384", 2023-01-25T12:30:58.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_caps_release_delay": "5", 2023-01-25T12:30:58.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_check_pool_perm": "true", 2023-01-25T12:30:58.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-25T12:30:58.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_force_sync_read": "false", 2023-01-25T12:30:58.052 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_getattr_caps": "false", 2023-01-25T12:30:58.052 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_inject_tick_delay": "0", 2023-01-25T12:30:58.052 INFO:teuthology.orchestra.run.smithi067.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-25T12:30:58.052 INFO:teuthology.orchestra.run.smithi067.stdout: "client_die_on_failed_remount": "false", 2023-01-25T12:30:58.052 INFO:teuthology.orchestra.run.smithi067.stdout: "client_dirsize_rbytes": "true", 2023-01-25T12:30:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_force_lazyio": "false", 2023-01-25T12:30:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_fs": "", 2023-01-25T12:30:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-25T12:30:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_inject_release_failure": "false", 2023-01-25T12:30:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_max_inline_size": "4096", 2023-01-25T12:30:58.054 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mds_namespace": "", 2023-01-25T12:30:58.054 INFO:teuthology.orchestra.run.smithi067.stdout: "client_metadata": "", 2023-01-25T12:30:58.054 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_gid": "-1", 2023-01-25T12:30:58.054 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_timeout": "300", 2023-01-25T12:30:58.054 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_uid": "-1", 2023-01-25T12:30:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mountpoint": "/", 2023-01-25T12:30:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client_notify_timeout": "10", 2023-01-25T12:30:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc": "true", 2023-01-25T12:30:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_dirty": "104857600", 2023-01-25T12:30:58.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-25T12:30:58.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_objects": "1000", 2023-01-25T12:30:58.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_size": "209715200", 2023-01-25T12:30:58.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_target_dirty": "8388608", 2023-01-25T12:30:58.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client_permissions": "true", 2023-01-25T12:30:58.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client_quota": "true", 2023-01-25T12:30:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_quota_df": "true", 2023-01-25T12:30:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_max_bytes": "0", 2023-01-25T12:30:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_max_periods": "4", 2023-01-25T12:30:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_min": "131072", 2023-01-25T12:30:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_reconnect_stale": "false", 2023-01-25T12:30:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "client_shutdown_timeout": "30", 2023-01-25T12:30:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "client_snapdir": ".snap", 2023-01-25T12:30:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "client_tick_interval": "1", 2023-01-25T12:30:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "client_trace": "", 2023-01-25T12:30:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "client_try_dentry_invalidate": "false", 2023-01-25T12:30:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_use_faked_inos": "false", 2023-01-25T12:30:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_use_random_mds": "false", 2023-01-25T12:30:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog": "false", 2023-01-25T12:30:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-25T12:30:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog_port": "12201", 2023-01-25T12:30:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_monitors": "default=true", 2023-01-25T12:30:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog": "false", 2023-01-25T12:30:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-25T12:30:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog_level": "info", 2023-01-25T12:30:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_addr": "-", 2023-01-25T12:30:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_network": "", 2023-01-25T12:30:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_network_interface": "", 2023-01-25T12:30:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_isal": "false", 2023-01-25T12:30:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_level": "5", 2023-01-25T12:30:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_winsize": "-15", 2023-01-25T12:30:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zstd_level": "1", 2023-01-25T12:30:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-25T12:30:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-25T12:30:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_alien_op_num_threads": "6", 2023-01-25T12:30:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-25T12:30:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-25T12:30:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-25T12:30:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_seastar_smp": "1", 2023-01-25T12:30:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "", 2023-01-25T12:30:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location_hook": "", 2023-01-25T12:30:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location_hook_timeout": "10", 2023-01-25T12:30:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "daemonize": "false", 2023-01-25T12:30:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "daos_pool": "tank", 2023-01-25T12:30:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-01-25T12:30:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-01-25T12:30:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_db_name_prefix": "dbstore", 2023-01-25T12:30:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_alienstore": "0/5", 2023-01-25T12:30:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_allow_any_pool_priority": "false", 2023-01-25T12:30:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asok": "1/5", 2023-01-25T12:30:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asok_assert_abort": "false", 2023-01-25T12:30:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asserts_on_shutdown": "true", 2023-01-25T12:30:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_auth": "20/20", 2023-01-25T12:30:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bdev": "1/3", 2023-01-25T12:30:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bluefs": "1/5", 2023-01-25T12:30:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bluestore": "1/5", 2023-01-25T12:30:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_buffer": "0/1", 2023-01-25T12:30:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_ceph_exporter": "1/5", 2023-01-25T12:30:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cephfs_mirror": "0/5", 2023-01-25T12:30:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cephsqlite": "0/5", 2023-01-25T12:30:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_client": "0/5", 2023-01-25T12:30:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_compressor": "1/5", 2023-01-25T12:30:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_context": "0/1", 2023-01-25T12:30:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_crush": "1/1", 2023-01-25T12:30:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_crypto": "1/5", 2023-01-25T12:30:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cyanstore": "0/5", 2023-01-25T12:30:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_deliberately_leak_memory": "false", 2023-01-25T12:30:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_disable_randomized_ping": "false", 2023-01-25T12:30:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_dpdk": "1/5", 2023-01-25T12:30:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_eventtrace": "1/5", 2023-01-25T12:30:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_filer": "0/1", 2023-01-25T12:30:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_filestore": "1/3", 2023-01-25T12:30:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_finisher": "1/1", 2023-01-25T12:30:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_fuse": "1/5", 2023-01-25T12:30:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_heartbeat_testing_span": "0", 2023-01-25T12:30:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_heartbeatmap": "1/5", 2023-01-25T12:30:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-25T12:30:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_javaclient": "1/5", 2023-01-25T12:30:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_journal": "1/3", 2023-01-25T12:30:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_journaler": "0/5", 2023-01-25T12:30:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_kstore": "1/5", 2023-01-25T12:30:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_leveldb": "4/5", 2023-01-25T12:30:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_lockdep": "0/1", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mclock": "1/5", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds": "1/5", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_balancer": "1/5", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_locker": "1/5", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_log": "1/5", 2023-01-25T12:30:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_log_expire": "1/5", 2023-01-25T12:30:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_migrator": "1/5", 2023-01-25T12:30:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_memstore": "1/5", 2023-01-25T12:30:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mgr": "2/5", 2023-01-25T12:30:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mgrc": "1/5", 2023-01-25T12:30:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mon": "20/20", 2023-01-25T12:30:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_monc": "0/10", 2023-01-25T12:30:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_ms": "1/1", 2023-01-25T12:30:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_none": "0/5", 2023-01-25T12:30:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objclass": "0/5", 2023-01-25T12:30:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objectcacher": "0/5", 2023-01-25T12:30:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objecter": "0/1", 2023-01-25T12:30:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_optracker": "0/5", 2023-01-25T12:30:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_osd": "1/5", 2023-01-25T12:30:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_paxos": "20/20", 2023-01-25T12:30:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_perfcounter": "1/5", 2023-01-25T12:30:58.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_prioritycache": "1/5", 2023-01-25T12:30:58.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rados": "0/5", 2023-01-25T12:30:58.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd": "0/5", 2023-01-25T12:30:58.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_mirror": "0/5", 2023-01-25T12:30:58.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_pwl": "0/5", 2023-01-25T12:30:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_replay": "0/5", 2023-01-25T12:30:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_refs": "0/0", 2023-01-25T12:30:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_reserver": "1/1", 2023-01-25T12:30:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw": "1/5", 2023-01-25T12:30:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_access": "1/5", 2023-01-25T12:30:58.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_datacache": "1/5", 2023-01-25T12:30:58.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_dbstore": "1/5", 2023-01-25T12:30:58.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_flight": "1/5", 2023-01-25T12:30:58.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_sync": "1/5", 2023-01-25T12:30:58.079 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rocksdb": "4/5", 2023-01-25T12:30:58.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore": "0/5", 2023-01-25T12:30:58.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_backref": "0/5", 2023-01-25T12:30:58.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_cache": "0/5", 2023-01-25T12:30:58.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_cleaner": "0/5", 2023-01-25T12:30:58.080 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_device": "0/5", 2023-01-25T12:30:58.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_epm": "0/5", 2023-01-25T12:30:58.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-01-25T12:30:58.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_journal": "0/5", 2023-01-25T12:30:58.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_lba": "0/5", 2023-01-25T12:30:58.081 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_odata": "0/5", 2023-01-25T12:30:58.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_omap": "0/5", 2023-01-25T12:30:58.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_onode": "0/5", 2023-01-25T12:30:58.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_t": "0/5", 2023-01-25T12:30:58.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_tm": "0/5", 2023-01-25T12:30:58.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_striper": "0/1", 2023-01-25T12:30:58.083 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_test": "0/5", 2023-01-25T12:30:58.083 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_throttle": "1/1", 2023-01-25T12:30:58.083 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_timer": "0/1", 2023-01-25T12:30:58.083 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_tp": "0/5", 2023-01-25T12:30:58.083 INFO:teuthology.orchestra.run.smithi067.stdout: "device_failure_prediction_mode": "none", 2023-01-25T12:30:58.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-25T12:30:58.084 INFO:teuthology.orchestra.run.smithi067.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-25T12:30:58.084 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_graylog": "false", 2023-01-25T12:30:58.084 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_journald": "false", 2023-01-25T12:30:58.085 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_stderr": "true", 2023-01-25T12:30:58.085 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_syslog": "false", 2023-01-25T12:30:58.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event_tracing": "false", 2023-01-25T12:30:58.085 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_addr": "0.0.0.0", 2023-01-25T12:30:58.085 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_http_port": "9926", 2023-01-25T12:30:58.086 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_prio_limit": "5", 2023-01-25T12:30:58.086 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-25T12:30:58.086 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_sort_metrics": "true", 2023-01-25T12:30:58.086 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_stats_period": "5", 2023-01-25T12:30:58.087 INFO:teuthology.orchestra.run.smithi067.stdout: "fake_statfs_for_testing": "0", 2023-01-25T12:30:58.087 INFO:teuthology.orchestra.run.smithi067.stdout: "fatal_signal_handlers": "true", 2023-01-25T12:30:58.087 INFO:teuthology.orchestra.run.smithi067.stdout: "filer_max_purge_ops": "10", 2023-01-25T12:30:58.087 INFO:teuthology.orchestra.run.smithi067.stdout: "filer_max_truncate_ops": "128", 2023-01-25T12:30:58.087 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_apply_finisher_threads": "1", 2023-01-25T12:30:58.088 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_blackhole": "false", 2023-01-25T12:30:58.088 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_btrfs_clone_range": "true", 2023-01-25T12:30:58.088 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_btrfs_snap": "true", 2023-01-25T12:30:58.088 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_caller_concurrency": "10", 2023-01-25T12:30:58.089 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_collect_device_partition_information": "true", 2023-01-25T12:30:58.089 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_commit_timeout": "600.000000", 2023-01-25T12:30:58.089 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_inject_read_err": "false", 2023-01-25T12:30:58.089 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_omap_check": "false", 2023-01-25T12:30:58.089 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-25T12:30:58.090 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_verify_split": "false", 2023-01-25T12:30:58.090 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_dump_file": "", 2023-01-25T12:30:58.090 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-25T12:30:58.090 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-25T12:30:58.090 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fadvise": "true", 2023-01-25T12:30:58.091 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fail_eio": "true", 2023-01-25T12:30:58.091 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fd_cache_shards": "16", 2023-01-25T12:30:58.091 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fd_cache_size": "128", 2023-01-25T12:30:58.091 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fiemap": "false", 2023-01-25T12:30:58.091 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fiemap_threshold": "4096", 2023-01-25T12:30:58.092 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-25T12:30:58.092 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-25T12:30:58.092 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_inject_stall": "0", 2023-01-25T12:30:58.093 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_parallel": "false", 2023-01-25T12:30:58.093 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_trailing": "false", 2023-01-25T12:30:58.093 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_writeahead": "false", 2023-01-25T12:30:58.093 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_kill_at": "0", 2023-01-25T12:30:58.093 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-25T12:30:58.094 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-25T12:30:58.094 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-25T12:30:58.094 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-25T12:30:58.094 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-25T12:30:58.094 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs": "0", 2023-01-25T12:30:58.095 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-25T12:30:58.095 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-25T12:30:58.095 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-25T12:30:58.095 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-25T12:30:58.095 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size": "0", 2023-01-25T12:30:58.096 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-25T12:30:58.096 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-25T12:30:58.096 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-25T12:30:58.096 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_merge_threshold": "-10", 2023-01-25T12:30:58.096 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-25T12:30:58.097 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_odsync_write": "false", 2023-01-25T12:30:58.097 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_backend": "rocksdb", 2023-01-25T12:30:58.097 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_backend_path": "", 2023-01-25T12:30:58.097 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-25T12:30:58.098 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-25T12:30:58.098 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-25T12:30:58.099 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_thread_timeout": "60", 2023-01-25T12:30:58.099 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_threads": "2", 2023-01-25T12:30:58.099 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_punch_hole": "false", 2023-01-25T12:30:58.099 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-25T12:30:58.099 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-25T12:30:58.100 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-25T12:30:58.100 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-25T12:30:58.100 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-25T12:30:58.100 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_ops": "50", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-25T12:30:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_seek_data_hole": "false", 2023-01-25T12:30:58.102 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_sloppy_crc": "false", 2023-01-25T12:30:58.102 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-25T12:30:58.102 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_splice": "false", 2023-01-25T12:30:58.102 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_split_multiple": "2", 2023-01-25T12:30:58.102 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_split_rand_factor": "20", 2023-01-25T12:30:58.103 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_update_to": "1000", 2023-01-25T12:30:58.103 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-25T12:30:58.104 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-25T12:30:58.104 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-25T12:30:58.104 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-25T12:30:58.105 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-25T12:30:58.105 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-25T12:30:58.105 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_enable": "true", 2023-01-25T12:30:58.105 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-25T12:30:58.105 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-25T12:30:58.106 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-25T12:30:58.106 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-25T12:30:58.106 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-25T12:30:58.106 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-25T12:30:58.106 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_xfs_extsize": "false", 2023-01-25T12:30:58.107 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_zfs_snap": "false", 2023-01-25T12:30:58.107 INFO:teuthology.orchestra.run.smithi067.stdout: "fio_dir": "/tmp/fio", 2023-01-25T12:30:58.107 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "b623256f-e4e6-420d-b240-c3bfae6a45f5", 2023-01-25T12:30:58.107 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_allow_other": "true", 2023-01-25T12:30:58.108 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_atomic_o_trunc": "true", 2023-01-25T12:30:58.108 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_big_writes": "true", 2023-01-25T12:30:58.108 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_debug": "false", 2023-01-25T12:30:58.108 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_default_permissions": "false", 2023-01-25T12:30:58.108 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_disable_pagecache": "false", 2023-01-25T12:30:58.109 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_max_write": "0", 2023-01-25T12:30:58.109 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_multithreaded": "true", 2023-01-25T12:30:58.109 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_require_active_mds": "true", 2023-01-25T12:30:58.109 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_set_user_groups": "true", 2023-01-25T12:30:58.109 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_move": "true", 2023-01-25T12:30:58.110 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_read": "true", 2023-01-25T12:30:58.110 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_write": "true", 2023-01-25T12:30:58.110 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-25T12:30:58.110 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_use_invalidate_cb": "true", 2023-01-25T12:30:58.110 INFO:teuthology.orchestra.run.smithi067.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-25T12:30:58.111 INFO:teuthology.orchestra.run.smithi067.stdout: "gss_target_name": "ceph", 2023-01-25T12:30:58.111 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_file": "", 2023-01-25T12:30:58.111 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_inject_failure": "0", 2023-01-25T12:30:58.111 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_interval": "5", 2023-01-25T12:30:58.112 INFO:teuthology.orchestra.run.smithi067.stdout: "host": "smithi067", 2023-01-25T12:30:58.112 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-25T12:30:58.112 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-25T12:30:58.112 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-25T12:30:58.112 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-25T12:30:58.113 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-25T12:30:58.113 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-25T12:30:58.113 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-25T12:30:58.113 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-25T12:30:58.113 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-25T12:30:58.114 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-25T12:30:58.114 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-25T12:30:58.114 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-25T12:30:58.114 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-25T12:30:58.114 INFO:teuthology.orchestra.run.smithi067.stdout: "inject_early_sigterm": "false", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "jaeger_tracing_enable": "false", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_aio": "true", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_align_min_size": "65536", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_block_align": "true", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_block_size": "4096", 2023-01-25T12:30:58.115 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_dio": "true", 2023-01-25T12:30:58.116 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_discard": "false", 2023-01-25T12:30:58.116 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_force_aio": "false", 2023-01-25T12:30:58.116 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_ignore_corruption": "false", 2023-01-25T12:30:58.116 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_max_write_bytes": "10485760", 2023-01-25T12:30:58.116 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_max_write_entries": "100", 2023-01-25T12:30:58.117 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_replay_from": "0", 2023-01-25T12:30:58.117 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-25T12:30:58.117 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-25T12:30:58.117 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-25T12:30:58.117 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-25T12:30:58.118 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_write_header_frequency": "0", 2023-01-25T12:30:58.118 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_zero_on_create": "false", 2023-01-25T12:30:58.118 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_prefetch_periods": "10", 2023-01-25T12:30:58.118 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_prezero_periods": "5", 2023-01-25T12:30:58.118 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_write_head_interval": "15", 2023-01-25T12:30:58.119 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "", 2023-01-25T12:30:58.119 INFO:teuthology.orchestra.run.smithi067.stdout: "keyfile": "", 2023-01-25T12:30:58.119 INFO:teuthology.orchestra.run.smithi067.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-25T12:30:58.119 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_backend": "rocksdb", 2023-01-25T12:30:58.119 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_default_stripe_size": "65536", 2023-01-25T12:30:58.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_fsck_on_mount": "false", 2023-01-25T12:30:58.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-25T12:30:58.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_max_bytes": "67108864", 2023-01-25T12:30:58.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_max_ops": "512", 2023-01-25T12:30:58.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_nid_prealloc": "1024", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_onode_map_size": "1024", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_sync_submit_transaction": "false", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_sync_transaction": "false", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_block_size": "65536", 2023-01-25T12:30:58.121 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_bloom_size": "0", 2023-01-25T12:30:58.122 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_cache_size": "536870912", 2023-01-25T12:30:58.122 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_compact_on_mount": "false", 2023-01-25T12:30:58.122 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_compression": "false", 2023-01-25T12:30:58.122 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_log": "", 2023-01-25T12:30:58.122 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-25T12:30:58.123 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_max_open_files": "0", 2023-01-25T12:30:58.123 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_paranoid": "false", 2023-01-25T12:30:58.123 INFO:teuthology.orchestra.run.smithi067.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-25T12:30:58.123 INFO:teuthology.orchestra.run.smithi067.stdout: "librados_thread_count": "2", 2023-01-25T12:30:58.123 INFO:teuthology.orchestra.run.smithi067.stdout: "lockdep": "false", 2023-01-25T12:30:58.124 INFO:teuthology.orchestra.run.smithi067.stdout: "lockdep_force_backtrace": "false", 2023-01-25T12:30:58.124 INFO:teuthology.orchestra.run.smithi067.stdout: "log_coarse_timestamps": "true", 2023-01-25T12:30:58.124 INFO:teuthology.orchestra.run.smithi067.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-25T12:30:58.124 INFO:teuthology.orchestra.run.smithi067.stdout: "log_flush_on_exit": "false", 2023-01-25T12:30:58.124 INFO:teuthology.orchestra.run.smithi067.stdout: "log_graylog_host": "127.0.0.1", 2023-01-25T12:30:58.125 INFO:teuthology.orchestra.run.smithi067.stdout: "log_graylog_port": "12201", 2023-01-25T12:30:58.125 INFO:teuthology.orchestra.run.smithi067.stdout: "log_max_new": "1000", 2023-01-25T12:30:58.125 INFO:teuthology.orchestra.run.smithi067.stdout: "log_max_recent": "10000", 2023-01-25T12:30:58.125 INFO:teuthology.orchestra.run.smithi067.stdout: "log_stderr_prefix": "", 2023-01-25T12:30:58.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_stop_at_utilization": "0.970000", 2023-01-25T12:30:58.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": "true", 2023-01-25T12:30:58.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_graylog": "false", 2023-01-25T12:30:58.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_journald": "false", 2023-01-25T12:30:58.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_stderr": "false", 2023-01-25T12:30:58.127 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_syslog": "false", 2023-01-25T12:30:58.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max_rotating_auth_attempts": "10", 2023-01-25T12:30:58.127 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_action_on_write_error": "1", 2023-01-25T12:30:58.127 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_alternate_name_max": "8192", 2023-01-25T12:30:58.127 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_asio_thread_count": "2", 2023-01-25T12:30:58.128 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_export_pin": "true", 2023-01-25T12:30:58.128 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_dirs": "true", 2023-01-25T12:30:58.128 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-25T12:30:58.128 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_interval": "5", 2023-01-25T12:30:58.128 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_interval": "10", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_max": "-1", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_max_until": "-1", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_merge_size": "50", 2023-01-25T12:30:58.129 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_midchunk": "0.300000", 2023-01-25T12:30:58.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-25T12:30:58.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_min_start": "0.200000", 2023-01-25T12:30:58.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_minchunk": "0.001000", 2023-01-25T12:30:58.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_mode": "0", 2023-01-25T12:30:58.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_need_max": "1.200000", 2023-01-25T12:30:58.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_need_min": "0.800000", 2023-01-25T12:30:58.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-25T12:30:58.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-25T12:30:58.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_bits": "3", 2023-01-25T12:30:58.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-25T12:30:58.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_size": "10000", 2023-01-25T12:30:58.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-25T12:30:58.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_target_decay": "10.000000", 2023-01-25T12:30:58.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-25T12:30:58.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_grace": "15.000000", 2023-01-25T12:30:58.133 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_interval": "4.000000", 2023-01-25T12:30:58.133 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-25T12:30:58.133 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-25T12:30:58.133 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_mid": "0.700000", 2023-01-25T12:30:58.133 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_release_free_interval": "10", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_reservation": "0.050000", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_interval": "1", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_threshold": "262144", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-25T12:30:58.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-25T12:30:58.135 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-25T12:30:58.135 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_prealloc_inos": "1000", 2023-01-25T12:30:58.135 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-25T12:30:58.135 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_connect_bootstrapping": "false", 2023-01-25T12:30:58.135 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_damage_table_max_entries": "10000", 2023-01-25T12:30:58.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-25T12:30:58.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_auth_pins": "false", 2023-01-25T12:30:58.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_frag": "false", 2023-01-25T12:30:58.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_scatterstat": "false", 2023-01-25T12:30:58.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_subtrees": "false", 2023-01-25T12:30:58.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_decay_halflife": "5.000000", 2023-01-25T12:30:58.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_default_dir_hash": "2", 2023-01-25T12:30:58.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_defer_session_stale": "true", 2023-01-25T12:30:58.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_deny_all_reconnect": "false", 2023-01-25T12:30:58.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_keys_per_op": "16384", 2023-01-25T12:30:58.138 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_max_commit_size": "10", 2023-01-25T12:30:58.138 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_max_entries": "0", 2023-01-25T12:30:58.138 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_prefetch": "true", 2023-01-25T12:30:58.138 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-25T12:30:58.138 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_on_map": "false", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_early_reply": "true", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_enable_op_tracker": "true", 2023-01-25T12:30:58.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_enforce_unique_name": "true", 2023-01-25T12:30:58.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-25T12:30:58.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-25T12:30:58.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_random": "true", 2023-01-25T12:30:58.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-25T12:30:58.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-01-25T12:30:58.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-25T12:30:58.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-25T12:30:58.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-01-25T12:30:58.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-25T12:30:58.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-25T12:30:58.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_health_summarize_threshold": "10", 2023-01-25T12:30:58.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-25T12:30:58.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-25T12:30:58.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_health_dummy": "false", 2023-01-25T12:30:58.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_migrator_session_race": "false", 2023-01-25T12:30:58.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-25T12:30:58.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_join_fs": "", 2023-01-25T12:30:58.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_journal_format": "1", 2023-01-25T12:30:58.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_create_at": "0", 2023-01-25T12:30:58.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_export_at": "0", 2023-01-25T12:30:58.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_import_at": "0", 2023-01-25T12:30:58.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_at": "0", 2023-01-25T12:30:58.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_expire_at": "0", 2023-01-25T12:30:58.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_replay_at": "0", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_link_at": "0", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_mdstable_at": "0", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_openc_at": "0", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_rename_at": "0", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_events_per_segment": "1024", 2023-01-25T12:30:58.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_max_events": "-1", 2023-01-25T12:30:58.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_max_segments": "128", 2023-01-25T12:30:58.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_pause": "false", 2023-01-25T12:30:58.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_segment_size": "0", 2023-01-25T12:30:58.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-25T12:30:58.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_warn_factor": "2.000000", 2023-01-25T12:30:58.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_caps_per_client": "1048576", 2023-01-25T12:30:58.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_completed_flushes": "100000", 2023-01-25T12:30:58.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_completed_requests": "100000", 2023-01-25T12:30:58.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_export_size": "20971520", 2023-01-25T12:30:58.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_file_recover": "32", 2023-01-25T12:30:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_files": "64", 2023-01-25T12:30:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_ops": "8192", 2023-01-25T12:30:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-25T12:30:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-25T12:30:58.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-25T12:30:58.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_snaps_per_dir": "100", 2023-01-25T12:30:58.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-25T12:30:58.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_metrics_update_interval": "2", 2023-01-25T12:30:58.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_min_caps_per_client": "100", 2023-01-25T12:30:58.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_min_caps_working_set": "10000", 2023-01-25T12:30:58.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-25T12:30:58.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_numa_node": "-1", 2023-01-25T12:30:58.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-25T12:30:58.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_complaint_time": "30.000000", 2023-01-25T12:30:58.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_duration": "600", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_size": "20", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_log_threshold": "5", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_ping_grace": "15", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_ping_interval": "5", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-25T12:30:58.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-25T12:30:58.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_caps": "30000", 2023-01-25T12:30:58.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-25T12:30:58.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-25T12:30:58.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-25T12:30:58.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_warning_threshold": "262144", 2023-01-25T12:30:58.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-25T12:30:58.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_replay_interval": "1.000000", 2023-01-25T12:30:58.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-25T12:30:58.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-25T12:30:58.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_root_ino_gid": "0", 2023-01-25T12:30:58.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_root_ino_uid": "0", 2023-01-25T12:30:58.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-25T12:30:58.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-25T12:30:58.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-25T12:30:58.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-25T12:30:58.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-25T12:30:58.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-25T12:30:58.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-25T12:30:58.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-25T12:30:58.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-25T12:30:58.156 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_shutdown_check": "0", 2023-01-25T12:30:58.156 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_skip_ino": "0", 2023-01-25T12:30:58.156 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-25T12:30:58.156 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_max_uid": "4294967294", 2023-01-25T12:30:58.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_min_uid": "0", 2023-01-25T12:30:58.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_rstat": "false", 2023-01-25T12:30:58.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_standby_replay_damaged": "false", 2023-01-25T12:30:58.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_symlink_recovery": "true", 2023-01-25T12:30:58.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-25T12:30:58.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_thrash_exports": "0", 2023-01-25T12:30:58.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_thrash_fragments": "0", 2023-01-25T12:30:58.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_tick_interval": "5.000000", 2023-01-25T12:30:58.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_valgrind_exit": "false", 2023-01-25T12:30:58.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_verify_backtrace": "1", 2023-01-25T12:30:58.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_verify_scatter": "false", 2023-01-25T12:30:58.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-25T12:30:58.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_wipe_sessions": "false", 2023-01-25T12:30:58.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool_debug": "false", 2023-01-25T12:30:58.160 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-25T12:30:58.160 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_device_bytes": "1073741824", 2023-01-25T12:30:58.160 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_page_set": "false", 2023-01-25T12:30:58.160 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_page_size": "65536", 2023-01-25T12:30:58.160 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_bytes": "134217728", 2023-01-25T12:30:58.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_messages": "512", 2023-01-25T12:30:58.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-25T12:30:58.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-25T12:30:58.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-25T12:30:58.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_disabled_modules": "", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_max_pg_num_change": "128", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mds_bytes": "134217728", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mds_messages": "128", 2023-01-25T12:30:58.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-25T12:30:58.163 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mon_bytes": "134217728", 2023-01-25T12:30:58.163 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mon_messages": "128", 2023-01-25T12:30:58.163 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_osd_bytes": "536870912", 2023-01-25T12:30:58.163 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_osd_messages": "8192", 2023-01-25T12:30:58.163 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_pool": "true", 2023-01-25T12:30:58.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-25T12:30:58.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_standby_modules": "true", 2023-01-25T12:30:58.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_stats_period": "5", 2023-01-25T12:30:58.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_stats_threshold": "5", 2023-01-25T12:30:58.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_tick_period": "2", 2023-01-25T12:30:58.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-25T12:30:58.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-25T12:30:58.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_allow_pool_delete": "true", 2023-01-25T12:30:58.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_allow_pool_size_one": "true", 2023-01-25T12:30:58.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_auth_validate_all_caps": "true", 2023-01-25T12:30:58.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-25T12:30:58.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-25T12:30:58.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_bytes": "104857600", 2023-01-25T12:30:58.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_directed_command_retry": "5", 2023-01-25T12:30:58.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-25T12:30:58.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-25T12:30:58.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-25T12:30:58.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-25T12:30:58.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_parallel": "3", 2023-01-25T12:30:58.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_log_interval": "1.000000", 2023-01-25T12:30:58.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-25T12:30:58.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_ping_interval": "10.000000", 2023-01-25T12:30:58.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-25T12:30:58.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-25T12:30:58.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_file_level": "debug", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_file": "true", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-25T12:30:58.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-25T12:30:58.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_journald": "false", 2023-01-25T12:30:58.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-25T12:30:58.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-25T12:30:58.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-25T12:30:58.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-25T12:30:58.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_bootstrap": "false", 2023-01-25T12:30:58.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_start": "false", 2023-01-25T12:30:58.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_trim": "true", 2023-01-25T12:30:58.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-25T12:30:58.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-25T12:30:58.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-25T12:30:58.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cpu_threads": "4", 2023-01-25T12:30:58.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_crush_min_required_version": "hammer", 2023-01-25T12:30:58.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_daemon_bytes": "419430400", 2023-01-25T12:30:58.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-25T12:30:58.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_avail_crit": "5", 2023-01-25T12:30:58.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_avail_warn": "5", 2023-01-25T12:30:58.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_size_warn": "16106127360", 2023-01-25T12:30:58.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-25T12:30:58.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_json": "false", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_transactions": "false", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_extra_checks": "false", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-25T12:30:58.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-25T12:30:58.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_quincy": "false", 2023-01-25T12:30:58.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_reef": "false", 2023-01-25T12:30:58.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-25T12:30:58.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-25T12:30:58.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-25T12:30:58.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_down_mkfs_grace": "120", 2023-01-25T12:30:58.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_election_default_strategy": "3", 2023-01-25T12:30:58.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_election_timeout": "5.000000", 2023-01-25T12:30:58.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-25T12:30:58.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ping_divisor": "2", 2023-01-25T12:30:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-25T12:30:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_enable_op_tracker": "true", 2023-01-25T12:30:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_fake_pool_delete": "false", 2023-01-25T12:30:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_force_quorum_join": "false", 2023-01-25T12:30:58.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_globalid_prealloc": "10000", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_detail_to_clog": "false", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_log_update_period": "5", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_max_detail": "50", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog": "true", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog_interval": "600", 2023-01-25T12:30:58.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-25T12:30:58.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_host": "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],172.21.15.87", 2023-01-25T12:30:58.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_host_override": "", 2023-01-25T12:30:58.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_initial_members": "", 2023-01-25T12:30:58.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-25T12:30:58.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-25T12:30:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-25T12:30:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-25T12:30:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-25T12:30:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease": "5.000000", 2023-01-25T12:30:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-25T12:30:58.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-25T12:30:58.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_full_interval": "50", 2023-01-25T12:30:58.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_max": "10000", 2023-01-25T12:30:58.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_max_summary": "50", 2023-01-25T12:30:58.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-25T12:30:58.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_log_epochs": "500", 2023-01-25T12:30:58.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-25T12:30:58.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-25T12:30:58.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_osd": "10000", 2023-01-25T12:30:58.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_pg_per_osd": "10000", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_pool_pg_num": "65536", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_force_trim_to": "0", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_skip_sanity": "false", 2023-01-25T12:30:58.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_memory_autotune": "true", 2023-01-25T12:30:58.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_memory_target": "2147483648", 2023-01-25T12:30:58.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_beacon_grace": "30", 2023-01-25T12:30:58.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-25T12:30:58.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_digest_period": "5", 2023-01-25T12:30:58.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_inactive_grace": "60", 2023-01-25T12:30:58.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-25T12:30:58.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-25T12:30:58.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_min_osdmap_epochs": "50", 2023-01-25T12:30:58.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_complaint_time": "30", 2023-01-25T12:30:58.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_duration": "600", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_size": "20", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_slow_op_size": "20", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_log_threshold": "5", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-25T12:30:58.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-25T12:30:58.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-25T12:30:58.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_in": "false", 2023-01-25T12:30:58.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-25T12:30:58.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-25T12:30:58.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-25T12:30:58.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_cache_size": "500", 2023-01-25T12:30:58.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-25T12:30:58.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-25T12:30:58.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-25T12:30:58.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_down_out_interval": "600", 2023-01-25T12:30:58.189 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-25T12:30:58.189 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-25T12:30:58.189 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_force_trim_to": "0", 2023-01-25T12:30:58.189 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-25T12:30:58.189 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-25T12:30:58.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-25T12:30:58.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-25T12:30:58.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-25T12:30:58.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-25T12:30:58.190 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-25T12:30:58.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-25T12:30:58.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_down_reporters": "2", 2023-01-25T12:30:58.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-25T12:30:58.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-25T12:30:58.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_report_timeout": "900", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-25T12:30:58.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-25T12:30:58.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-25T12:30:58.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-25T12:30:58.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-25T12:30:58.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-25T12:30:58.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-25T12:30:58.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-25T12:30:58.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-25T12:30:58.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_stuck_threshold": "60", 2023-01-25T12:30:58.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-25T12:30:58.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-25T12:30:58.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-25T12:30:58.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-25T12:30:58.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-25T12:30:58.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-25T12:30:58.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_probe_timeout": "2.000000", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_max_change": "0.050000", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_max_osds": "4", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-25T12:30:58.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-25T12:30:58.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-25T12:30:58.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_interval": "300", 2023-01-25T12:30:58.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_max_keys": "100", 2023-01-25T12:30:58.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_timeout": "300", 2023-01-25T12:30:58.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_session_timeout": "300", 2023-01-25T12:30:58.198 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_smart_report_timeout": "5", 2023-01-25T12:30:58.198 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stat_smooth_intervals": "6", 2023-01-25T12:30:58.198 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-25T12:30:58.198 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_pool_min_size": "2", 2023-01-25T12:30:58.198 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_pool_size": "4", 2023-01-25T12:30:58.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-25T12:30:58.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-25T12:30:58.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_debug": "false", 2023-01-25T12:30:58.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-25T12:30:58.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-25T12:30:58.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_provider_kill_at": "0", 2023-01-25T12:30:58.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_requester_kill_at": "0", 2023-01-25T12:30:58.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_timeout": "60.000000", 2023-01-25T12:30:58.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_target_pg_per_osd": "100", 2023-01-25T12:30:58.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_tick_interval": "5", 2023-01-25T12:30:58.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_timecheck_interval": "300.000000", 2023-01-25T12:30:58.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-25T12:30:58.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_use_min_delay_socket": "false", 2023-01-25T12:30:58.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_older_version_delay": "604800", 2023-01-25T12:30:58.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-25T12:30:58.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-25T12:30:58.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_misplaced": "false", 2023-01-25T12:30:58.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-25T12:30:58.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_older_version": "true", 2023-01-25T12:30:58.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-25T12:30:58.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-25T12:30:58.204 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-25T12:30:58.204 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-25T12:30:58.204 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-25T12:30:58.204 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_