2023-01-24T18:40:49.393 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T18:40:49.394 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607 --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607/orig.config.yaml 2023-01-24T18:40:49.440 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T18:40:49.502 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} email: yweinstein@radhat.com first_in_suite: false job_id: '7135607' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 - machine: ram: 15000 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon election default strategy: 1 ms type: async osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd inject bad map crc probability: 0.1 osd map cache size: 1 osd max backfills: 6 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub during recovery: false osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - but it is still running - objects unfound and apparently lost - osd_map_cache_size - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgTaFh/6b/8RAKwHz8RfCnQGB5WOUdECya1GkbHQWNxTansiR8PKNwfkeiDpFmpwp0i0eANT1/DFgcPY/bvLeNocGcObszm6DKMXB3R76sCVF5bihjs/LC7b7sA3TBCXrsolh0X5c4PxmpXbZnxekz+V0vpshYpne+wuTKDbiWr39Gl8jfXs44UV/oDEMFbX+3+npHMEVQUlFhCoCTeJpPk+D0eLMPlKwHOfVTYGTUThQ2Mdlc4zmi5XGh5kPWS4RyIcC/0NjhKTSiQeL7OkGl1G/4ccI5ztT5p29ZPFoJNtOFz7qc0LeRD3PG98XNgP6oo1IrO1yKmrZ1tdVOCRxSJ69GOciwNBe+cYVtLl1NVkPLUMg/sCvETH+ZCvtNwOf6MfrtYZDu2Qd/6OXsceEUu6kIqLPC5jMAbdfAfvPWt61Oqks4aRR2Ry1sB3u0zmu2KhjmurQ4N8+S4sFkNXAFTOrwqwLQWf8y6pH1x8NM7R/P3RcRoW+z2DCBxeuKQo8= smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NJNFqlUBucOXzFl8K9PLwkCJiVbyzGxrpb3vFXa5SjZG9rTRSW0HyN+BgfVYvgb8nV0lhKoZ4CXZvVRBzuAoF/+8CV91bXg7zF0COIj5ESs7WWYcwETDwt0xZjLLY+CC9DTy1qzN5ZFAHAA46+cfHGZ1Sz3VBjxSXQ0l8ElfufPY59B96kJxmL8+FMwaHNYIL/ZQ1YU9nCmvhkDrUYv7D/0eZ6NIFNN+WntrSZxl+eKO/e7Z4VIzh3gHBS7KFMb69Bg8gh0wUToax/JfBV4A8JpbuHEqjF8uqttKhO9IovqScLaYxij4Ou6TTicBvgn75iYj+P4XSi+Vnzkhk/q28RWli7bLhJEF7lWKia1WxPmLgow27W656Cv/bxroxSdiM1w71fvJCb2xKd1pvilPPKQk/Wkn5Hgtr5UgOSWQSHlYcTE6ENEsj8ixj5Cf3K5zQg5r18sBb9Nx+pdIgkMyheqeJ2T5kiPcpi/oDO1lrJvdiAIFtjRYkuVRVH1P31E= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 0.25 chance_pgnum_shrink: 0.25 chance_pgpnum_fix: 0.25 chance_test_map_discontinuity: 2 timeout: 1800 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 0 - sudo ceph osd pool set cache min_write_recency_for_promote 0 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T18:40:49.502 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T18:40:49.503 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T18:40:49.504 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T18:40:49.506 INFO:teuthology.task.internal:Checking packages... 2023-01-24T18:40:49.536 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T18:40:49.536 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T18:40:49.537 INFO:teuthology.packaging:ref: None 2023-01-24T18:40:49.537 INFO:teuthology.packaging:tag: None 2023-01-24T18:40:49.537 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:40:49.538 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:40:49.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:40:49.687 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T18:40:49.687 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T18:40:49.697 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T18:40:49.697 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T18:40:49.731 INFO:teuthology.task.internal:Saving configuration 2023-01-24T18:40:49.756 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T18:40:49.773 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T18:40:49.796 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:35:27.040460', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCqkI5nxBaqTc2AnSZsnBkqsf/Li0R1gZp9+Nevigjze78jcHJsyfRWebzjR8u/KyLW0ZYhY5K6QXUyM9pOLPY='} 2023-01-24T18:40:49.819 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:35:27.039087', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLL48yZ2g5Vyb7SSfWzEjXnaiRorQuKqrWK2vyxtah/cyHyjVge4bV3vAFeojCOPhPz1ybe1eMiNpHjwiMgBT1s='} 2023-01-24T18:40:49.819 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T18:40:49.830 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T18:40:49.830 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T18:40:49.830 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T18:40:49.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20636715e0>, signals=[15]) 2023-01-24T18:40:49.924 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T18:40:49.932 INFO:teuthology.task.internal:Opening connections... 2023-01-24T18:40:49.933 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:40:49.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:40:50.006 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:40:50.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:40:50.078 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T18:40:50.085 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2023-01-24T18:40:50.103 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2023-01-24T18:40:50.103 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-01-24T18:40:50.160 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T18:40:50.160 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T18:40:50.160 INFO:teuthology.orchestra.run.smithi145.stdout:ID="rhel" 2023-01-24T18:40:50.161 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="fedora" 2023-01-24T18:40:50.161 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8.4" 2023-01-24T18:40:50.161 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-01-24T18:40:50.162 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T18:40:50.162 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-01-24T18:40:50.162 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T18:40:50.162 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T18:40:50.162 INFO:teuthology.orchestra.run.smithi145.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T18:40:50.163 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T18:40:50.163 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:40:50.163 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T18:40:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T18:40:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T18:40:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T18:40:50.166 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-01-24T18:40:50.182 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-01-24T18:40:50.199 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-01-24T18:40:50.200 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-01-24T18:40:50.260 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T18:40:50.261 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T18:40:50.261 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2023-01-24T18:40:50.261 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2023-01-24T18:40:50.262 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.4" 2023-01-24T18:40:50.262 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-01-24T18:40:50.262 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T18:40:50.262 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-01-24T18:40:50.263 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T18:40:50.263 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T18:40:50.263 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T18:40:50.263 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T18:40:50.263 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:40:50.264 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T18:40:50.264 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T18:40:50.264 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T18:40:50.264 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T18:40:50.265 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-01-24T18:40:50.278 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T18:40:50.289 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T18:40:50.315 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T18:40:50.315 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2023-01-24T18:40:50.318 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-01-24T18:40:50.334 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T18:40:50.364 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T18:40:50.364 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2023-01-24T18:40:50.374 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-01-24T18:40:50.387 INFO:teuthology.orchestra.run.smithi145.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T18:40:50.393 INFO:teuthology.orchestra.run.smithi167.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T18:40:50.394 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T18:40:50.449 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T18:40:50.466 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T18:40:50.466 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T18:40:50.467 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T18:40:50.468 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T18:40:50.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:40:50.483 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2023-01-24T18:40:50.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:40:50.485 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-01-24T18:40:50.539 INFO:teuthology.orchestra.run.smithi145.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:40:50.539 INFO:teuthology.task.kernel:Running kernel on smithi145: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:40:50.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2023-01-24T18:40:50.544 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:40:50.544 INFO:teuthology.task.kernel:Running kernel on smithi167: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:40:50.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2023-01-24T18:40:52.015 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:52.017 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2023-01-24T18:40:52.075 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:52.075 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2023-01-24T18:40:53.392 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-01-24T18:40:53.400 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-01-24T18:40:54.636 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-24T18:40:54.642 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-24T18:40:57.393 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-01-24T18:40:57.396 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-01-24T18:40:58.044 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T18:40:58.045 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T18:40:58.045 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T18:40:58.080 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T18:40:58.080 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T18:40:58.081 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T18:40:58.173 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:40:58.173 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:40:58.174 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:40:58.202 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:40:58.203 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:40:58.204 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:40:58.276 DEBUG:teuthology.orchestra.run.smithi145:> echo no | sudo yum reinstall kernel || true 2023-01-24T18:40:58.302 DEBUG:teuthology.orchestra.run.smithi167:> echo no | sudo yum reinstall kernel || true 2023-01-24T18:40:58.620 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:58.620 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2023-01-24T18:40:58.647 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:58.647 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2023-01-24T18:40:58.901 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:01 ago on Tue 24 Jan 2023 06:40:57 PM UTC. 2023-01-24T18:40:58.930 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:01 ago on Tue 24 Jan 2023 06:40:57 PM UTC. 2023-01-24T18:40:59.041 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:40:59.041 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.041 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.068 INFO:teuthology.orchestra.run.smithi145.stderr:Error: No packages marked for reinstall. 2023-01-24T18:40:59.073 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:40:59.074 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.074 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.102 INFO:teuthology.orchestra.run.smithi167.stderr:Error: No packages marked for reinstall. 2023-01-24T18:40:59.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum reinstall -y kernel || true 2023-01-24T18:40:59.152 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum reinstall -y kernel || true 2023-01-24T18:40:59.460 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:59.460 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2023-01-24T18:40:59.495 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:40:59.495 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2023-01-24T18:40:59.736 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 06:40:57 PM UTC. 2023-01-24T18:40:59.778 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 06:40:57 PM UTC. 2023-01-24T18:40:59.873 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:40:59.874 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.874 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.890 INFO:teuthology.orchestra.run.smithi145.stderr:Error: No packages marked for reinstall. 2023-01-24T18:40:59.915 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:40:59.915 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.915 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:40:59.936 INFO:teuthology.orchestra.run.smithi167.stderr:Error: No packages marked for reinstall. 2023-01-24T18:40:59.939 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel --last | head -n 1 2023-01-24T18:40:59.985 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel --last | head -n 1 2023-01-24T18:41:00.065 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T18:41:00.067 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:41:00.067 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T18:41:00.067 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T18:41:00.067 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T18:41:00.097 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2023-01-24T18:41:00.109 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T18:41:00.111 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:41:00.111 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T18:41:00.111 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T18:41:00.111 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T18:41:00.113 DEBUG:teuthology.parallel:result is None 2023-01-24T18:41:00.142 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2023-01-24T18:41:00.155 DEBUG:teuthology.parallel:result is None 2023-01-24T18:41:00.155 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T18:41:00.162 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T18:41:00.163 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T18:41:00.166 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T18:41:00.208 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T18:41:00.215 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T18:41:00.241 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T18:41:00.241 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T18:41:00.244 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T18:41:00.277 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T18:41:00.288 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T18:41:00.288 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T18:41:00.314 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T18:41:00.348 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:41:00.349 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:41:00.357 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:41:00.359 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:41:00.360 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T18:41:00.372 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T18:41:00.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T18:41:00.402 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T18:41:00.425 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T18:41:00.436 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T18:41:00.437 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T18:41:00.466 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T18:41:00.480 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:41:00.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:41:00.598 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:41:00.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:41:00.718 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:41:00.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T18:41:00.780 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:41:00.823 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:41:00.863 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:41:00.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:41:00.983 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:41:00.983 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T18:41:01.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2023-01-24T18:41:01.047 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-01-24T18:41:01.096 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T18:41:01.141 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T18:41:01.483 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T18:41:01.491 INFO:teuthology.task.internal:Starting timer... 2023-01-24T18:41:01.492 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T18:41:01.524 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T18:41:01.556 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T18:41:01.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2023-01-24T18:41:01.613 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: [ OK ] 2023-01-24T18:41:01.614 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2023-01-24T18:41:01.681 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: [ OK ] 2023-01-24T18:41:01.682 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T18:41:01.683 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2023-01-24T18:41:01.705 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2023-01-24T18:41:01.705 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2023-01-24T18:41:01.722 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2023-01-24T18:41:01.723 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi145.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2023-01-24T18:41:01.723 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T18:41:01.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:41:01.770 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T18:41:01.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:41:01.796 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T18:41:01.796 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2023-01-24T18:41:01.850 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2023-01-24T18:41:01.875 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T18:41:01.885 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-24T18:41:01.885 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T18:41:01.902 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T18:41:01.903 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi145.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T18:47:26.257 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi145.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-01-24T18:47:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2023-01-24T18:47:26.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:47:26.330 DEBUG:teuthology.orchestra.run.smithi145:> true 2023-01-24T18:47:26.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-01-24T18:47:26.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:47:26.463 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-01-24T18:47:26.531 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T18:47:26.542 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T18:47:26.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T18:47:26.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T18:47:26.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T18:47:26.547 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T18:47:26.573 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T18:47:26.588 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T18:47:26.600 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T18:47:26.616 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T18:47:26.621 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2023-01-24T18:47:26.631 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2023-01-24T18:47:26.644 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2023-01-24T18:47:26.645 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T18:47:26.654 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2023-01-24T18:47:26.658 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T18:47:26.667 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T18:47:26.680 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T18:47:26.730 INFO:teuthology.orchestra.run.smithi145.stderr:bash: ntpq: command not found 2023-01-24T18:47:26.733 INFO:teuthology.orchestra.run.smithi145.stdout:210 Number of sources = 4 2023-01-24T18:47:26.733 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T18:47:26.733 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2023-01-24T18:47:26.733 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.734 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.734 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.734 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-01-24T18:47:26.752 INFO:teuthology.orchestra.run.smithi167.stdout:210 Number of sources = 4 2023-01-24T18:47:26.752 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T18:47:26.753 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-01-24T18:47:26.753 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.753 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.753 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.754 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:47:26.755 INFO:teuthology.run_tasks:Running task install... 2023-01-24T18:47:26.763 DEBUG:teuthology.task.install:project ceph 2023-01-24T18:47:26.764 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T18:47:26.764 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T18:47:26.764 INFO:teuthology.task.install:Using flavor: default 2023-01-24T18:47:26.771 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T18:47:26.771 INFO:teuthology.task.install:extra packages: [] 2023-01-24T18:47:26.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2023-01-24T18:47:26.810 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2023-01-24T18:47:26.811 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:47:26.811 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2023-01-24T18:47:26.812 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T18:47:26.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2023-01-24T18:47:26.850 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2023-01-24T18:47:26.851 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:47:26.851 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T18:47:26.852 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2023-01-24T18:47:26.889 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2023-01-24T18:47:26.890 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:47:26.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:47:26.929 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2023-01-24T18:47:26.930 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:47:26.930 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:47:27.043 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T18:47:27.043 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T18:47:27.079 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T18:47:27.079 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T18:47:27.160 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T18:47:27.160 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:47:27.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T18:47:27.184 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T18:47:27.185 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T18:47:27.198 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T18:47:27.199 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:47:27.199 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T18:47:27.223 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T18:47:27.224 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T18:47:27.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T18:47:27.288 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T18:47:27.323 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T18:47:27.362 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T18:47:27.386 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2023-01-24T18:47:27.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2023-01-24T18:47:27.425 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2023-01-24T18:47:27.427 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2023-01-24T18:47:27.771 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:47:27.812 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:47:28.082 INFO:teuthology.orchestra.run.smithi145.stdout:44 files removed 2023-01-24T18:47:28.131 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw 2023-01-24T18:47:28.132 INFO:teuthology.orchestra.run.smithi167.stdout:44 files removed 2023-01-24T18:47:28.181 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw 2023-01-24T18:47:28.476 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:47:28.525 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:47:29.193 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 273 kB/s | 78 kB 00:00 2023-01-24T18:47:29.252 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 258 kB/s | 78 kB 00:00 2023-01-24T18:47:29.453 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 42 kB/s | 9.2 kB 00:00 2023-01-24T18:47:29.512 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 42 kB/s | 9.2 kB 00:00 2023-01-24T18:47:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 6.6 kB/s | 1.5 kB 00:00 2023-01-24T18:47:29.758 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-01-24T18:47:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 47 kB/s | 3.5 kB 00:00 2023-01-24T18:47:29.868 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-01-24T18:47:30.805 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-24T18:47:30.870 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-24T18:47:33.632 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-01-24T18:47:33.715 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 4.9 MB/s | 24 kB 00:00 2023-01-24T18:47:33.998 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 132 MB/s | 38 MB 00:00 2023-01-24T18:47:34.096 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 126 MB/s | 38 MB 00:00 2023-01-24T18:47:39.639 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 MB/s | 33 MB 00:00 2023-01-24T18:47:39.732 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 125 MB/s | 33 MB 00:00 2023-01-24T18:47:48.020 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:47:48.023 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================== 2023-01-24T18:47:48.023 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:47:48.023 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================== 2023-01-24T18:47:48.023 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:47:48.024 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:47:48.024 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2023-01-24T18:47:48.024 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:47:48.024 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T18:47:48.025 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:47:48.025 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T18:47:48.025 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T18:47:48.026 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T18:47:48.026 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T18:47:48.026 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T18:47:48.026 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T18:47:48.027 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T18:47:48.027 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T18:47:48.027 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T18:47:48.027 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T18:47:48.028 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T18:47:48.028 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T18:47:48.028 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T18:47:48.028 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T18:47:48.028 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T18:47:48.029 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T18:47:48.029 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T18:47:48.029 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T18:47:48.029 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T18:47:48.029 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T18:47:48.030 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:47:48.030 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:47:48.030 INFO:teuthology.orchestra.run.smithi167.stdout:=================================================================================================== 2023-01-24T18:47:48.030 INFO:teuthology.orchestra.run.smithi167.stdout:Install 21 Packages 2023-01-24T18:47:48.030 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2023-01-24T18:47:48.031 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:47:48.031 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 63 M 2023-01-24T18:47:48.032 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:47:48.260 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:47:48.262 INFO:teuthology.orchestra.run.smithi145.stdout:=================================================================================================== 2023-01-24T18:47:48.263 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:47:48.263 INFO:teuthology.orchestra.run.smithi145.stdout:=================================================================================================== 2023-01-24T18:47:48.263 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:47:48.263 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:47:48.264 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2023-01-24T18:47:48.264 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:47:48.264 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T18:47:48.264 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:47:48.265 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T18:47:48.265 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T18:47:48.265 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T18:47:48.265 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T18:47:48.266 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T18:47:48.266 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T18:47:48.266 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T18:47:48.266 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T18:47:48.267 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T18:47:48.267 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T18:47:48.267 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T18:47:48.267 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T18:47:48.267 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T18:47:48.268 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T18:47:48.268 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T18:47:48.268 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T18:47:48.268 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T18:47:48.269 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T18:47:48.269 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T18:47:48.269 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T18:47:48.269 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:47:48.269 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:47:48.270 INFO:teuthology.orchestra.run.smithi145.stdout:=================================================================================================== 2023-01-24T18:47:48.270 INFO:teuthology.orchestra.run.smithi145.stdout:Install 21 Packages 2023-01-24T18:47:48.270 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 2 Packages 2023-01-24T18:47:48.270 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:47:48.271 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 63 M 2023-01-24T18:47:48.271 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:47:49.092 INFO:teuthology.orchestra.run.smithi167.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 4.8 MB/s | 5.1 MB 00:01 2023-01-24T18:47:49.159 INFO:teuthology.orchestra.run.smithi167.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 359 kB/s | 24 kB 00:00 2023-01-24T18:47:49.409 INFO:teuthology.orchestra.run.smithi167.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 3.1 MB/s | 793 kB 00:00 2023-01-24T18:47:49.533 INFO:teuthology.orchestra.run.smithi167.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 4.0 MB/s | 506 kB 00:00 2023-01-24T18:47:50.263 INFO:teuthology.orchestra.run.smithi145.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 2.6 MB/s | 5.1 MB 00:01 2023-01-24T18:47:50.329 INFO:teuthology.orchestra.run.smithi145.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 371 kB/s | 24 kB 00:00 2023-01-24T18:47:50.620 INFO:teuthology.orchestra.run.smithi145.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 2.7 MB/s | 793 kB 00:00 2023-01-24T18:47:50.837 INFO:teuthology.orchestra.run.smithi145.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 2.3 MB/s | 506 kB 00:00 2023-01-24T18:47:50.844 INFO:teuthology.orchestra.run.smithi167.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 4.2 MB/s | 5.5 MB 00:01 2023-01-24T18:47:50.900 INFO:teuthology.orchestra.run.smithi167.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 811 kB/s | 45 kB 00:00 2023-01-24T18:47:50.939 INFO:teuthology.orchestra.run.smithi167.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 2.7 MB/s | 108 kB 00:00 2023-01-24T18:47:50.982 INFO:teuthology.orchestra.run.smithi167.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 4.3 MB/s | 190 kB 00:00 2023-01-24T18:47:50.990 INFO:teuthology.orchestra.run.smithi145.stdout:(5/23): ceph-common-17.2.5-727.g49f8fb05.el8.x8 8.5 MB/s | 23 MB 00:02 2023-01-24T18:47:51.049 INFO:teuthology.orchestra.run.smithi145.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 774 kB/s | 45 kB 00:00 2023-01-24T18:47:51.070 INFO:teuthology.orchestra.run.smithi167.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 4.1 MB/s | 370 kB 00:00 2023-01-24T18:47:51.098 INFO:teuthology.orchestra.run.smithi145.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 2.1 MB/s | 108 kB 00:00 2023-01-24T18:47:51.160 INFO:teuthology.orchestra.run.smithi145.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 3.0 MB/s | 190 kB 00:00 2023-01-24T18:47:51.180 INFO:teuthology.orchestra.run.smithi167.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 3.2 MB/s | 357 kB 00:00 2023-01-24T18:47:51.219 INFO:teuthology.orchestra.run.smithi145.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 6.2 MB/s | 370 kB 00:00 2023-01-24T18:47:51.241 INFO:teuthology.orchestra.run.smithi167.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 1.7 MB/s | 107 kB 00:00 2023-01-24T18:47:51.281 INFO:teuthology.orchestra.run.smithi145.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 5.6 MB/s | 357 kB 00:00 2023-01-24T18:47:51.330 INFO:teuthology.orchestra.run.smithi145.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 2.1 MB/s | 107 kB 00:00 2023-01-24T18:47:51.433 INFO:teuthology.orchestra.run.smithi167.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-01-24T18:47:51.461 INFO:teuthology.orchestra.run.smithi167.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 2.1 MB/s | 59 kB 00:00 2023-01-24T18:47:51.490 INFO:teuthology.orchestra.run.smithi167.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 2.6 MB/s | 75 kB 00:00 2023-01-24T18:47:51.493 INFO:teuthology.orchestra.run.smithi145.stdout:(12/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 5.2 MB/s | 17 MB 00:03 2023-01-24T18:47:51.540 INFO:teuthology.orchestra.run.smithi145.stdout:(13/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-01-24T18:47:51.569 INFO:teuthology.orchestra.run.smithi145.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 2.6 MB/s | 75 kB 00:00 2023-01-24T18:47:51.609 INFO:teuthology.orchestra.run.smithi145.stdout:(15/23): liboath-2.6.2-3.el8.x86_64.rpm 510 kB/s | 59 kB 00:00 2023-01-24T18:47:51.636 INFO:teuthology.orchestra.run.smithi145.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.7 MB/s | 47 kB 00:00 2023-01-24T18:47:51.647 INFO:teuthology.orchestra.run.smithi145.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 3.7 MB/s | 44 kB 00:00 2023-01-24T18:47:51.661 INFO:teuthology.orchestra.run.smithi145.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 19 MB/s | 259 kB 00:00 2023-01-24T18:47:51.671 INFO:teuthology.orchestra.run.smithi167.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 9.6 MB/s | 1.7 MB 00:00 2023-01-24T18:47:51.678 INFO:teuthology.orchestra.run.smithi145.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 22 MB/s | 353 kB 00:00 2023-01-24T18:47:51.699 INFO:teuthology.orchestra.run.smithi167.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.6 MB/s | 47 kB 00:00 2023-01-24T18:47:51.710 INFO:teuthology.orchestra.run.smithi167.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 3.9 MB/s | 44 kB 00:00 2023-01-24T18:47:51.724 INFO:teuthology.orchestra.run.smithi167.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 19 MB/s | 259 kB 00:00 2023-01-24T18:47:51.739 INFO:teuthology.orchestra.run.smithi167.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 24 MB/s | 353 kB 00:00 2023-01-24T18:47:51.746 INFO:teuthology.orchestra.run.smithi145.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 9.8 MB/s | 1.7 MB 00:00 2023-01-24T18:47:52.055 INFO:teuthology.orchestra.run.smithi145.stdout:(21/23): librados2-17.2.5-727.g49f8fb05.el8.x86 9.2 MB/s | 3.5 MB 00:00 2023-01-24T18:47:52.172 INFO:teuthology.orchestra.run.smithi145.stdout:(22/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_6 4.1 MB/s | 5.5 MB 00:01 2023-01-24T18:47:52.237 INFO:teuthology.orchestra.run.smithi167.stdout:(20/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 5.5 MB/s | 23 MB 00:04 2023-01-24T18:47:52.257 INFO:teuthology.orchestra.run.smithi145.stdout:(23/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 7.0 MB/s | 3.6 MB 00:00 2023-01-24T18:47:52.259 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:47:52.260 INFO:teuthology.orchestra.run.smithi145.stdout:Total 16 MB/s | 63 MB 00:03 2023-01-24T18:47:52.263 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:47:52.306 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:47:52.307 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:47:52.357 INFO:teuthology.orchestra.run.smithi167.stdout:(21/23): librados2-17.2.5-727.g49f8fb05.el8.x86 5.6 MB/s | 3.5 MB 00:00 2023-01-24T18:47:52.517 INFO:teuthology.orchestra.run.smithi167.stdout:(22/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 3.7 MB/s | 17 MB 00:04 2023-01-24T18:47:52.615 INFO:teuthology.orchestra.run.smithi167.stdout:(23/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 9.5 MB/s | 3.6 MB 00:00 2023-01-24T18:47:52.618 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:47:52.618 INFO:teuthology.orchestra.run.smithi167.stdout:Total 14 MB/s | 63 MB 00:04 2023-01-24T18:47:52.621 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:47:52.665 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:47:52.665 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:47:53.058 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:47:53.092 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:47:53.414 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:47:53.418 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:47:54.221 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:47:54.409 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:47:54.564 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:47:54.771 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:47:54.808 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:47:55.059 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:47:55.147 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:47:55.157 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:47:55.373 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:47:55.388 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:47:55.466 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:47:55.667 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T18:47:55.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:47:55.901 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:47:55.946 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T18:47:56.163 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:47:56.305 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:47:56.501 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:47:56.561 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:47:56.631 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:47:56.729 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:47:56.800 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:47:56.826 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:47:56.855 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:47:57.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:47:57.115 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:47:57.202 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T18:47:57.431 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T18:47:57.873 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T18:47:58.075 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:47:58.121 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T18:47:58.143 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:47:58.252 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T18:47:58.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:47:58.388 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:47:58.461 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:47:58.514 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T18:47:58.718 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:47:58.724 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:47:58.932 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T18:47:58.947 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:47:59.108 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:47:59.182 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:47:59.191 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T18:47:59.337 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:47:59.368 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T18:47:59.410 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:47:59.560 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T18:47:59.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T18:47:59.780 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T18:47:59.835 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T18:48:00.026 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T18:48:00.057 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T18:48:00.238 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T18:48:04.684 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:04.916 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:04.992 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:05.220 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:05.494 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:05.786 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:48:05.793 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:48:05.889 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:48:05.999 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:06.102 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:48:06.183 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:48:06.303 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:23.852 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:23.853 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2023-01-24T18:48:23.853 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2023-01-24T18:48:23.853 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2023-01-24T18:48:23.853 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /run 2023-01-24T18:48:23.853 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2023-01-24T18:48:23.854 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2023-01-24T18:48:23.854 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2023-01-24T18:48:23.854 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2023-01-24T18:48:23.854 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2023-01-24T18:48:23.855 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:23.987 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2023-01-24T18:48:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2023-01-24T18:48:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-01-24T18:48:23.988 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2023-01-24T18:48:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2023-01-24T18:48:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2023-01-24T18:48:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2023-01-24T18:48:23.989 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2023-01-24T18:48:23.990 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-01-24T18:48:23.990 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:25.590 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:48:25.654 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:48:25.654 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:48:25.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:48:25.776 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:48:25.776 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:48:25.872 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:48:25.872 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:25.994 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:48:25.994 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:26.389 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:26.390 INFO:teuthology.orchestra.run.smithi145.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T18:48:26.390 INFO:teuthology.orchestra.run.smithi145.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T18:48:26.390 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:26.494 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:26.494 INFO:teuthology.orchestra.run.smithi167.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T18:48:26.494 INFO:teuthology.orchestra.run.smithi167.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T18:48:26.495 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:26.680 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T18:48:26.681 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:48:26.681 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:48:26.681 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T18:48:26.681 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:48:26.682 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:48:26.682 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T18:48:26.682 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T18:48:26.683 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T18:48:26.683 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:48:26.683 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T18:48:26.683 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:48:26.684 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:48:26.684 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T18:48:26.684 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T18:48:26.685 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T18:48:26.685 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T18:48:26.686 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-24T18:48:26.686 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-24T18:48:26.686 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2023-01-24T18:48:26.686 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2023-01-24T18:48:26.687 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:26.687 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T18:48:26.687 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T18:48:26.797 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T18:48:26.797 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:48:26.798 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:48:26.798 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T18:48:26.798 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:48:26.798 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:48:26.798 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T18:48:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T18:48:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T18:48:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:48:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T18:48:26.800 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:48:26.800 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:48:26.800 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T18:48:26.800 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T18:48:26.800 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T18:48:26.801 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T18:48:26.801 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-24T18:48:26.801 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-24T18:48:26.801 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2023-01-24T18:48:26.802 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2023-01-24T18:48:26.802 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:48:26.802 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T18:48:26.802 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T18:48:28.058 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:28.059 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:48:28.174 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:30.287 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:48:30.364 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:48:30.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:30.401 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2023-01-24T18:48:30.401 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.401 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.401 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:48:30.402 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.402 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.402 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.402 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.402 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T18:48:30.403 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.403 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T18:48:30.403 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T18:48:30.403 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.404 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T18:48:30.404 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.404 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T18:48:30.405 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T18:48:30.405 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.405 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.405 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.405 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T18:48:30.406 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.406 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.406 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.406 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T18:48:30.407 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:30.407 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:48:30.470 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:30.471 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2023-01-24T18:48:30.471 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.471 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.471 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:48:30.472 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.472 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.472 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.472 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.472 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T18:48:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T18:48:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T18:48:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T18:48:30.474 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.474 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T18:48:30.474 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T18:48:30.474 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.474 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.475 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.475 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T18:48:30.475 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.475 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.475 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:30.476 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T18:48:30.476 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:30.476 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:48:30.537 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-test 2023-01-24T18:48:30.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-test 2023-01-24T18:48:30.895 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:48:30.972 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:48:31.673 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:31.733 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:52 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:33.430 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:48:33.431 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================== 2023-01-24T18:48:33.431 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:48:33.431 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================== 2023-01-24T18:48:33.432 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:48:33.432 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T18:48:33.432 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:48:33.432 INFO:teuthology.orchestra.run.smithi145.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T18:48:33.433 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T18:48:33.433 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T18:48:33.433 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T18:48:33.433 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T18:48:33.433 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:33.434 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:48:33.434 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================== 2023-01-24T18:48:33.434 INFO:teuthology.orchestra.run.smithi145.stdout:Install 6 Packages 2023-01-24T18:48:33.434 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:33.435 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 53 M 2023-01-24T18:48:33.435 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 233 M 2023-01-24T18:48:33.435 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:48:33.436 INFO:teuthology.orchestra.run.smithi145.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 22 MB/s | 70 kB 00:00 2023-01-24T18:48:33.468 INFO:teuthology.orchestra.run.smithi145.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.1 MB/s | 161 kB 00:00 2023-01-24T18:48:33.482 INFO:teuthology.orchestra.run.smithi145.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 13 MB/s | 187 kB 00:00 2023-01-24T18:48:33.494 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:48:33.495 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================== 2023-01-24T18:48:33.495 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:48:33.495 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================== 2023-01-24T18:48:33.495 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:48:33.495 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T18:48:33.496 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:48:33.496 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T18:48:33.496 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T18:48:33.496 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T18:48:33.497 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T18:48:33.497 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T18:48:33.498 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:33.498 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:48:33.498 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================== 2023-01-24T18:48:33.498 INFO:teuthology.orchestra.run.smithi167.stdout:Install 6 Packages 2023-01-24T18:48:33.499 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:33.499 INFO:teuthology.orchestra.run.smithi145.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 21 MB/s | 302 kB 00:00 2023-01-24T18:48:33.500 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 53 M 2023-01-24T18:48:33.500 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 233 M 2023-01-24T18:48:33.500 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:48:33.501 INFO:teuthology.orchestra.run.smithi167.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 22 MB/s | 70 kB 00:00 2023-01-24T18:48:33.532 INFO:teuthology.orchestra.run.smithi167.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.0 MB/s | 161 kB 00:00 2023-01-24T18:48:33.545 INFO:teuthology.orchestra.run.smithi167.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 14 MB/s | 187 kB 00:00 2023-01-24T18:48:33.559 INFO:teuthology.orchestra.run.smithi167.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 22 MB/s | 302 kB 00:00 2023-01-24T18:48:33.626 INFO:teuthology.orchestra.run.smithi145.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 904 kB/s | 174 kB 00:00 2023-01-24T18:48:33.692 INFO:teuthology.orchestra.run.smithi167.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 893 kB/s | 174 kB 00:00 2023-01-24T18:48:38.054 INFO:teuthology.orchestra.run.smithi145.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 11 MB/s | 52 MB 00:04 2023-01-24T18:48:38.056 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:48:38.057 INFO:teuthology.orchestra.run.smithi145.stdout:Total 11 MB/s | 53 MB 00:04 2023-01-24T18:48:38.061 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:48:38.084 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:48:38.084 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:48:38.205 INFO:teuthology.orchestra.run.smithi167.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 11 MB/s | 52 MB 00:04 2023-01-24T18:48:38.208 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:48:38.208 INFO:teuthology.orchestra.run.smithi167.stdout:Total 11 MB/s | 53 MB 00:04 2023-01-24T18:48:38.212 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:48:38.235 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:48:38.235 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:48:38.465 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:48:38.469 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:48:38.584 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:48:38.589 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:48:39.020 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:48:39.125 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:48:39.214 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T18:48:39.312 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T18:48:39.434 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:48:39.513 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:48:39.516 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:48:39.594 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:48:39.668 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T18:48:39.770 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T18:48:39.866 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T18:48:39.963 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T18:48:40.000 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:48:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:48:45.048 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:48:45.067 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:48:45.307 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:48:45.330 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:48:45.558 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:48:45.558 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T18:48:45.559 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T18:48:45.559 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T18:48:45.559 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T18:48:45.559 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T18:48:45.599 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:48:45.600 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T18:48:45.601 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T18:48:45.601 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T18:48:45.601 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T18:48:45.602 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T18:48:46.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T18:48:46.542 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:46.593 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T18:48:46.594 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:48.705 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:48:48.737 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:48:48.802 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:48.803 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:48:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: jq-1.5-12.el8.x86_64 2023-01-24T18:48:48.803 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T18:48:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T18:48:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T18:48:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:48.804 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:48:48.836 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:48.837 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:48:48.837 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:48.837 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.5-12.el8.x86_64 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:48.838 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:48:48.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph 2023-01-24T18:48:48.955 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph 2023-01-24T18:48:49.269 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:48:49.301 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:48:50.028 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:11 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:50.068 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:11 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:48:51.813 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:48:51.816 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================== 2023-01-24T18:48:51.816 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:48:51.816 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================== 2023-01-24T18:48:51.817 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:48:51.817 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T18:48:51.817 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:48:51.817 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T18:48:51.817 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T18:48:51.818 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T18:48:51.818 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T18:48:51.818 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:48:51.818 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T18:48:51.818 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T18:48:51.819 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T18:48:51.819 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T18:48:51.819 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T18:48:51.819 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T18:48:51.819 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T18:48:51.820 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T18:48:51.820 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T18:48:51.820 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T18:48:51.820 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T18:48:51.820 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T18:48:51.821 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T18:48:51.821 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T18:48:51.821 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T18:48:51.821 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T18:48:51.821 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T18:48:51.822 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T18:48:51.822 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T18:48:51.822 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T18:48:51.822 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T18:48:51.823 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T18:48:51.823 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:51.823 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:48:51.824 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================== 2023-01-24T18:48:51.824 INFO:teuthology.orchestra.run.smithi145.stdout:Install 28 Packages 2023-01-24T18:48:51.824 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:48:51.825 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 27 M 2023-01-24T18:48:51.826 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 105 M 2023-01-24T18:48:51.826 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:48:51.837 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:48:51.840 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================== 2023-01-24T18:48:51.840 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:48:51.840 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================== 2023-01-24T18:48:51.841 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:48:51.841 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T18:48:51.842 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:48:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T18:48:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T18:48:51.842 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T18:48:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T18:48:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:48:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T18:48:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T18:48:51.843 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T18:48:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T18:48:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T18:48:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T18:48:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T18:48:51.844 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T18:48:51.845 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T18:48:51.845 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T18:48:51.845 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T18:48:51.845 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T18:48:51.845 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T18:48:51.846 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T18:48:51.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T18:48:51.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T18:48:51.847 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T18:48:51.847 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:51.847 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:48:51.848 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================== 2023-01-24T18:48:51.848 INFO:teuthology.orchestra.run.smithi167.stdout:Install 28 Packages 2023-01-24T18:48:51.848 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:48:51.849 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 27 M 2023-01-24T18:48:51.849 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 105 M 2023-01-24T18:48:51.849 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:48:51.920 INFO:teuthology.orchestra.run.smithi145.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 64 kB/s | 6.4 kB 00:00 2023-01-24T18:48:51.943 INFO:teuthology.orchestra.run.smithi167.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 65 kB/s | 6.4 kB 00:00 2023-01-24T18:48:52.276 INFO:teuthology.orchestra.run.smithi145.stdout:(2/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 5.0 MB/s | 2.3 MB 00:00 2023-01-24T18:48:52.372 INFO:teuthology.orchestra.run.smithi167.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 3.0 MB/s | 1.6 MB 00:00 2023-01-24T18:48:52.519 INFO:teuthology.orchestra.run.smithi145.stdout:(3/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 6.6 MB/s | 3.9 MB 00:00 2023-01-24T18:48:52.523 INFO:teuthology.orchestra.run.smithi145.stdout:(4/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 2.2 MB/s | 1.6 MB 00:00 2023-01-24T18:48:52.571 INFO:teuthology.orchestra.run.smithi145.stdout:(5/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 4.4 MB/s | 233 kB 00:00 2023-01-24T18:48:52.617 INFO:teuthology.orchestra.run.smithi167.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 3.0 MB/s | 2.3 MB 00:00 2023-01-24T18:48:52.637 INFO:teuthology.orchestra.run.smithi145.stdout:(6/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 386 kB/s | 44 kB 00:00 2023-01-24T18:48:52.702 INFO:teuthology.orchestra.run.smithi167.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 2.7 MB/s | 233 kB 00:00 2023-01-24T18:48:52.708 INFO:teuthology.orchestra.run.smithi145.stdout:(7/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.3 MB/s | 185 kB 00:00 2023-01-24T18:48:52.721 INFO:teuthology.orchestra.run.smithi145.stdout:(8/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.0 MB/s | 173 kB 00:00 2023-01-24T18:48:52.750 INFO:teuthology.orchestra.run.smithi145.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 391 kB/s | 11 kB 00:00 2023-01-24T18:48:52.765 INFO:teuthology.orchestra.run.smithi145.stdout:(10/28): python3-cherrypy-18.4.0-1.el8.noarch.r 6.6 MB/s | 384 kB 00:00 2023-01-24T18:48:52.778 INFO:teuthology.orchestra.run.smithi145.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 623 kB/s | 18 kB 00:00 2023-01-24T18:48:52.794 INFO:teuthology.orchestra.run.smithi145.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.7 MB/s | 49 kB 00:00 2023-01-24T18:48:52.807 INFO:teuthology.orchestra.run.smithi145.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2023-01-24T18:48:52.808 INFO:teuthology.orchestra.run.smithi167.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 409 kB/s | 44 kB 00:00 2023-01-24T18:48:52.835 INFO:teuthology.orchestra.run.smithi145.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 555 kB/s | 16 kB 00:00 2023-01-24T18:48:52.849 INFO:teuthology.orchestra.run.smithi145.stdout:(15/28): python3-pecan-1.3.2-9.el8.noarch.rpm 5.0 MB/s | 283 kB 00:00 2023-01-24T18:48:52.864 INFO:teuthology.orchestra.run.smithi145.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 661 kB/s | 19 kB 00:00 2023-01-24T18:48:52.877 INFO:teuthology.orchestra.run.smithi145.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 869 kB/s | 24 kB 00:00 2023-01-24T18:48:52.888 INFO:teuthology.orchestra.run.smithi167.stdout:(6/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.3 MB/s | 185 kB 00:00 2023-01-24T18:48:52.892 INFO:teuthology.orchestra.run.smithi145.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T18:48:52.905 INFO:teuthology.orchestra.run.smithi145.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 986 kB/s | 27 kB 00:00 2023-01-24T18:48:52.917 INFO:teuthology.orchestra.run.smithi167.stdout:(7/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 5.9 MB/s | 173 kB 00:00 2023-01-24T18:48:52.936 INFO:teuthology.orchestra.run.smithi145.stdout:(20/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 8.1 MB/s | 251 kB 00:00 2023-01-24T18:48:52.949 INFO:teuthology.orchestra.run.smithi145.stdout:(21/28): python3-waitress-1.2.1-2.el8.1.noarch. 4.1 MB/s | 241 kB 00:00 2023-01-24T18:48:52.965 INFO:teuthology.orchestra.run.smithi145.stdout:(22/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.9 MB/s | 85 kB 00:00 2023-01-24T18:48:52.972 INFO:teuthology.orchestra.run.smithi167.stdout:(8/28): python3-cherrypy-18.4.0-1.el8.noarch.rp 6.8 MB/s | 384 kB 00:00 2023-01-24T18:48:52.979 INFO:teuthology.orchestra.run.smithi145.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 779 kB/s | 23 kB 00:00 2023-01-24T18:48:52.999 INFO:teuthology.orchestra.run.smithi167.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 417 kB/s | 11 kB 00:00 2023-01-24T18:48:53.000 INFO:teuthology.orchestra.run.smithi145.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 13 MB/s | 457 kB 00:00 2023-01-24T18:48:53.009 INFO:teuthology.orchestra.run.smithi145.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.3 MB/s | 39 kB 00:00 2023-01-24T18:48:53.018 INFO:teuthology.orchestra.run.smithi145.stdout:(26/28): python3-mako-1.0.6-13.el8.noarch.rpm 8.7 MB/s | 157 kB 00:00 2023-01-24T18:48:53.024 INFO:teuthology.orchestra.run.smithi145.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-01-24T18:48:53.026 INFO:teuthology.orchestra.run.smithi167.stdout:(10/28): python3-jaraco-functools-2.0-4.el8.noa 656 kB/s | 18 kB 00:00 2023-01-24T18:48:53.039 INFO:teuthology.orchestra.run.smithi167.stdout:(11/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_ 3.6 MB/s | 3.9 MB 00:01 2023-01-24T18:48:53.053 INFO:teuthology.orchestra.run.smithi167.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.8 MB/s | 49 kB 00:00 2023-01-24T18:48:53.107 INFO:teuthology.orchestra.run.smithi167.stdout:(13/28): python3-pecan-1.3.2-9.el8.noarch.rpm 5.2 MB/s | 283 kB 00:00 2023-01-24T18:48:53.133 INFO:teuthology.orchestra.run.smithi167.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 588 kB/s | 16 kB 00:00 2023-01-24T18:48:53.151 INFO:teuthology.orchestra.run.smithi167.stdout:(15/28): python3-more-itertools-7.2.0-3.el8.noa 523 kB/s | 59 kB 00:00 2023-01-24T18:48:53.160 INFO:teuthology.orchestra.run.smithi167.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 702 kB/s | 19 kB 00:00 2023-01-24T18:48:53.180 INFO:teuthology.orchestra.run.smithi167.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 850 kB/s | 24 kB 00:00 2023-01-24T18:48:53.187 INFO:teuthology.orchestra.run.smithi167.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.1 MB/s | 29 kB 00:00 2023-01-24T18:48:53.208 INFO:teuthology.orchestra.run.smithi167.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 976 kB/s | 27 kB 00:00 2023-01-24T18:48:53.216 INFO:teuthology.orchestra.run.smithi167.stdout:(20/28): python3-waitress-1.2.1-2.el8.1.noarch. 8.1 MB/s | 241 kB 00:00 2023-01-24T18:48:53.244 INFO:teuthology.orchestra.run.smithi167.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 3.1 MB/s | 85 kB 00:00 2023-01-24T18:48:53.271 INFO:teuthology.orchestra.run.smithi167.stdout:(22/28): python3-zc-lockfile-2.0-2.el8.noarch.r 856 kB/s | 23 kB 00:00 2023-01-24T18:48:53.293 INFO:teuthology.orchestra.run.smithi167.stdout:(23/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.9 MB/s | 251 kB 00:00 2023-01-24T18:48:53.313 INFO:teuthology.orchestra.run.smithi167.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 11 MB/s | 457 kB 00:00 2023-01-24T18:48:53.324 INFO:teuthology.orchestra.run.smithi167.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.2 MB/s | 39 kB 00:00 2023-01-24T18:48:53.332 INFO:teuthology.orchestra.run.smithi167.stdout:(26/28): python3-mako-1.0.6-13.el8.noarch.rpm 8.7 MB/s | 157 kB 00:00 2023-01-24T18:48:53.339 INFO:teuthology.orchestra.run.smithi167.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-01-24T18:48:53.753 INFO:teuthology.orchestra.run.smithi145.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 11 MB/s | 17 MB 00:01 2023-01-24T18:48:53.756 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:48:53.757 INFO:teuthology.orchestra.run.smithi145.stdout:Total 14 MB/s | 27 MB 00:01 2023-01-24T18:48:53.761 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:48:53.796 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:48:53.797 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:48:53.815 INFO:teuthology.orchestra.run.smithi167.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 12 MB/s | 17 MB 00:01 2023-01-24T18:48:53.818 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:48:53.818 INFO:teuthology.orchestra.run.smithi167.stdout:Total 14 MB/s | 27 MB 00:01 2023-01-24T18:48:53.822 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:48:53.858 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:48:53.858 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:48:54.182 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:48:54.187 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:48:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:48:54.232 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:48:55.279 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:48:55.352 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:48:55.488 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T18:48:55.565 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T18:48:55.736 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T18:48:55.794 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T18:48:55.900 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T18:48:55.943 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T18:48:56.169 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T18:48:56.212 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T18:48:56.508 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T18:48:56.572 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T18:48:56.713 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T18:48:56.823 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T18:48:56.963 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T18:48:57.032 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T18:48:57.162 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T18:48:57.256 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T18:48:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T18:48:57.396 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T18:48:57.492 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T18:48:57.561 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T18:48:57.706 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T18:48:57.726 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T18:48:57.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T18:48:57.934 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T18:48:58.071 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T18:48:58.085 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T18:48:58.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T18:48:58.317 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T18:48:58.545 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T18:48:58.721 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T18:48:59.178 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T18:48:59.345 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T18:48:59.513 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T18:48:59.620 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T18:48:59.771 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T18:48:59.880 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T18:49:00.025 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T18:49:00.091 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T18:49:00.309 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T18:49:00.429 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T18:49:00.531 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T18:49:00.714 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T18:49:00.825 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T18:49:01.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T18:49:01.199 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T18:49:01.364 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T18:49:01.375 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:49:01.569 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:49:02.861 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:49:03.050 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:49:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:49:03.244 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:49:03.452 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:49:03.643 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:49:03.657 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:49:03.856 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:49:03.874 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:49:04.038 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:49:04.055 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:49:04.072 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:49:04.154 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:49:04.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:49:04.229 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:49:04.327 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:49:04.538 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:49:04.538 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T18:49:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T18:49:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T18:49:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T18:49:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T18:49:04.539 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T18:49:04.540 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T18:49:04.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T18:49:04.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T18:49:04.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T18:49:04.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T18:49:04.541 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T18:49:04.542 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T18:49:04.542 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T18:49:04.542 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T18:49:04.542 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T18:49:04.542 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T18:49:04.543 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T18:49:04.543 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T18:49:04.543 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T18:49:04.543 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T18:49:04.543 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T18:49:04.544 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T18:49:04.544 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T18:49:04.544 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T18:49:04.545 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T18:49:04.545 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T18:49:04.740 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:49:04.740 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T18:49:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T18:49:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T18:49:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T18:49:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T18:49:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T18:49:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T18:49:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T18:49:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T18:49:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T18:49:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T18:49:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T18:49:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T18:49:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T18:49:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T18:49:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T18:49:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T18:49:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T18:49:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T18:49:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T18:49:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T18:49:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T18:49:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T18:49:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T18:49:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T18:49:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T18:49:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T18:49:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T18:49:06.115 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:06.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T18:49:06.250 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:27 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:08.207 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:49:08.291 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:49:08.306 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:08.306 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:49:08.307 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.307 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.307 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.308 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:08.308 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.308 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.308 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T18:49:08.309 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T18:49:08.309 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T18:49:08.309 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T18:49:08.309 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T18:49:08.310 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T18:49:08.310 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T18:49:08.310 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T18:49:08.310 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T18:49:08.311 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T18:49:08.311 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T18:49:08.311 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T18:49:08.312 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T18:49:08.312 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T18:49:08.312 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T18:49:08.312 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T18:49:08.313 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T18:49:08.313 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T18:49:08.313 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T18:49:08.314 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T18:49:08.314 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T18:49:08.314 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T18:49:08.314 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:08.314 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:08.390 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:08.390 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:49:08.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.391 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.391 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.391 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:08.391 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.392 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:08.392 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T18:49:08.392 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T18:49:08.392 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T18:49:08.392 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T18:49:08.393 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T18:49:08.393 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T18:49:08.393 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T18:49:08.393 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T18:49:08.393 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T18:49:08.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T18:49:08.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T18:49:08.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T18:49:08.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T18:49:08.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T18:49:08.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T18:49:08.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T18:49:08.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T18:49:08.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T18:49:08.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T18:49:08.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T18:49:08.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T18:49:08.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T18:49:08.396 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:08.396 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:08.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-base 2023-01-24T18:49:08.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-base 2023-01-24T18:49:08.836 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:08.893 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:09.588 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:30 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:09.644 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:30 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:10.885 INFO:teuthology.orchestra.run.smithi145.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:49:10.960 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:49:11.364 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:11.365 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:49:11.365 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:11.413 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:11.414 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:49:11.414 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:11.464 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install cephadm 2023-01-24T18:49:11.519 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install cephadm 2023-01-24T18:49:11.819 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:11.879 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:12.585 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:12.624 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:33 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:14.348 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:14.349 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:14.350 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:49:14.350 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:14.350 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:49:14.350 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T18:49:14.350 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:14.351 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:49:14.351 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:14.351 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:49:14.351 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:14.351 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 78 k 2023-01-24T18:49:14.352 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 355 k 2023-01-24T18:49:14.352 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:49:14.446 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:14.447 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:14.447 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:49:14.448 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:14.448 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:49:14.448 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T18:49:14.448 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:14.449 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:49:14.449 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:14.449 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:49:14.449 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:14.449 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 78 k 2023-01-24T18:49:14.450 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 355 k 2023-01-24T18:49:14.450 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:49:14.495 INFO:teuthology.orchestra.run.smithi145.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 542 kB/s | 78 kB 00:00 2023-01-24T18:49:14.496 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:14.497 INFO:teuthology.orchestra.run.smithi145.stdout:Total 537 kB/s | 78 kB 00:00 2023-01-24T18:49:14.498 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:49:14.504 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:49:14.504 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:49:14.505 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:49:14.512 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:49:14.552 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:49:14.593 INFO:teuthology.orchestra.run.smithi167.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 542 kB/s | 78 kB 00:00 2023-01-24T18:49:14.593 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:14.594 INFO:teuthology.orchestra.run.smithi167.stdout:Total 537 kB/s | 78 kB 00:00 2023-01-24T18:49:14.594 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:49:14.602 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:49:14.602 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:49:14.604 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:49:14.608 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:49:14.652 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:49:19.111 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:19.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:19.371 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:19.557 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:19.578 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:20.479 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:20.479 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:41 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:20.508 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:49:20.509 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:41 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:22.651 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:49:22.713 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:49:22.747 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:22.748 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:49:22.748 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:22.748 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:22.748 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:22.809 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:22.810 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:49:22.810 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:22.810 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:22.811 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:22.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T18:49:22.981 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T18:49:23.250 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:23.984 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:44 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:24.110 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:45 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:25.763 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:25.764 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:25.764 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2023-01-24T18:49:25.764 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:25.764 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:49:25.765 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T18:49:25.765 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:25.765 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:49:25.765 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:49:25.766 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:49:25.766 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:25.766 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 162 k 2023-01-24T18:49:25.766 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 490 k 2023-01-24T18:49:25.766 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:49:25.882 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:25.883 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:25.883 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-01-24T18:49:25.883 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:25.883 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:49:25.883 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T18:49:25.884 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:25.884 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:49:25.884 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:49:25.884 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:49:25.885 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:25.885 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 162 k 2023-01-24T18:49:25.885 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 490 k 2023-01-24T18:49:25.885 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:49:25.930 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 976 kB/s | 162 kB 00:00 2023-01-24T18:49:25.931 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:25.932 INFO:teuthology.orchestra.run.smithi145.stdout:Total 967 kB/s | 162 kB 00:00 2023-01-24T18:49:25.932 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:49:25.944 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:49:25.944 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:49:25.998 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:49:26.002 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:49:26.049 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 975 kB/s | 162 kB 00:00 2023-01-24T18:49:26.050 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:26.051 INFO:teuthology.orchestra.run.smithi167.stdout:Total 963 kB/s | 162 kB 00:00 2023-01-24T18:49:26.052 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:49:26.064 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:49:26.064 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:49:26.116 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:49:26.120 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:49:26.138 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:49:26.262 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:49:26.317 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:26.433 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:26.718 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:26.836 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:27.614 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:27.614 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:27.747 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:49:27.747 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:49:29.804 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:49:29.833 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:49:29.834 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:29.834 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:29.900 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:29.901 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:49:29.901 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:49:29.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:29.901 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:29.954 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr 2023-01-24T18:49:30.025 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr 2023-01-24T18:49:30.311 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:30.385 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:31.074 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:31.138 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:32.409 INFO:teuthology.orchestra.run.smithi145.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:49:32.494 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:49:32.862 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:32.863 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:49:32.863 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:32.952 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:32.953 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:49:32.953 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:32.961 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T18:49:33.061 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T18:49:33.321 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:33.422 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:34.112 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:34.215 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:35.876 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:35.877 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================================== 2023-01-24T18:49:35.878 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:49:35.878 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================================== 2023-01-24T18:49:35.878 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:49:35.878 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T18:49:35.878 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:49:35.879 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T18:49:35.879 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T18:49:35.879 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:49:35.879 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T18:49:35.879 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T18:49:35.880 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:35.880 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:49:35.880 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================================== 2023-01-24T18:49:35.880 INFO:teuthology.orchestra.run.smithi145.stdout:Install 6 Packages 2023-01-24T18:49:35.880 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:35.881 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 2.0 M 2023-01-24T18:49:35.884 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 7.8 M 2023-01-24T18:49:35.884 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:49:36.000 INFO:teuthology.orchestra.run.smithi145.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 190 kB/s | 23 kB 00:00 2023-01-24T18:49:36.010 INFO:teuthology.orchestra.run.smithi145.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 112 kB/s | 14 kB 00:00 2023-01-24T18:49:36.045 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:36.046 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================== 2023-01-24T18:49:36.047 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:49:36.047 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================== 2023-01-24T18:49:36.047 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:49:36.047 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T18:49:36.048 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:49:36.048 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T18:49:36.048 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T18:49:36.049 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:49:36.049 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T18:49:36.050 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T18:49:36.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:36.050 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:49:36.050 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================== 2023-01-24T18:49:36.051 INFO:teuthology.orchestra.run.smithi167.stdout:Install 6 Packages 2023-01-24T18:49:36.051 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:36.051 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 2.0 M 2023-01-24T18:49:36.052 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 7.8 M 2023-01-24T18:49:36.052 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:49:36.084 INFO:teuthology.orchestra.run.smithi145.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 402 kB/s | 34 kB 00:00 2023-01-24T18:49:36.115 INFO:teuthology.orchestra.run.smithi145.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.4 MB/s | 43 kB 00:00 2023-01-24T18:49:36.146 INFO:teuthology.orchestra.run.smithi145.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.4 MB/s | 196 kB 00:00 2023-01-24T18:49:36.152 INFO:teuthology.orchestra.run.smithi145.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 6.1 MB/s | 1.6 MB 00:00 2023-01-24T18:49:36.154 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:36.154 INFO:teuthology.orchestra.run.smithi145.stdout:Total 7.1 MB/s | 2.0 MB 00:00 2023-01-24T18:49:36.156 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:49:36.164 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:49:36.164 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:49:36.174 INFO:teuthology.orchestra.run.smithi167.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 183 kB/s | 23 kB 00:00 2023-01-24T18:49:36.179 INFO:teuthology.orchestra.run.smithi167.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 111 kB/s | 14 kB 00:00 2023-01-24T18:49:36.243 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:49:36.248 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:49:36.257 INFO:teuthology.orchestra.run.smithi167.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 407 kB/s | 34 kB 00:00 2023-01-24T18:49:36.288 INFO:teuthology.orchestra.run.smithi167.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.4 MB/s | 43 kB 00:00 2023-01-24T18:49:36.314 INFO:teuthology.orchestra.run.smithi167.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.4 MB/s | 196 kB 00:00 2023-01-24T18:49:36.319 INFO:teuthology.orchestra.run.smithi167.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 6.1 MB/s | 1.6 MB 00:00 2023-01-24T18:49:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:36.322 INFO:teuthology.orchestra.run.smithi167.stdout:Total 7.1 MB/s | 2.0 MB 00:00 2023-01-24T18:49:36.324 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:49:36.331 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:49:36.332 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:49:36.420 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:49:36.424 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:49:36.525 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:49:36.708 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:49:36.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T18:49:36.924 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T18:49:36.949 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T18:49:37.101 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T18:49:37.157 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T18:49:37.213 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T18:49:37.367 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T18:49:37.496 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T18:49:37.597 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T18:49:37.842 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T18:49:37.843 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:49:38.113 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:49:38.134 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:49:38.134 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T18:49:38.135 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T18:49:38.135 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T18:49:38.135 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T18:49:38.135 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T18:49:38.358 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:49:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T18:49:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T18:49:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T18:49:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T18:49:38.360 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T18:49:39.038 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T18:49:39.038 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:00 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:39.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T18:49:39.257 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:00 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:40.993 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:49:41.093 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:41.093 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:49:41.093 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.094 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.094 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.094 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T18:49:41.094 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T18:49:41.094 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T18:49:41.095 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:41.095 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:41.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T18:49:41.231 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:49:41.327 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:41.328 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:49:41.328 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.328 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.328 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:41.328 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T18:49:41.329 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T18:49:41.329 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T18:49:41.329 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:41.329 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:41.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T18:49:41.568 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:41.854 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:42.318 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:03 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:42.602 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:03 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:44.152 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:44.154 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================================== 2023-01-24T18:49:44.154 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:49:44.156 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================================== 2023-01-24T18:49:44.156 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:49:44.156 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T18:49:44.157 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:49:44.157 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T18:49:44.157 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T18:49:44.157 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T18:49:44.157 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T18:49:44.158 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T18:49:44.158 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T18:49:44.158 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T18:49:44.158 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:44.159 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:49:44.159 INFO:teuthology.orchestra.run.smithi145.stdout:====================================================================================================================== 2023-01-24T18:49:44.159 INFO:teuthology.orchestra.run.smithi145.stdout:Install 8 Packages 2023-01-24T18:49:44.159 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:44.160 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 36 M 2023-01-24T18:49:44.160 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 199 M 2023-01-24T18:49:44.160 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:49:44.192 INFO:teuthology.orchestra.run.smithi145.stdout:(1/8): libgfortran-8.4.1-1.el8.x86_64.rpm 18 MB/s | 639 kB 00:00 2023-01-24T18:49:44.192 INFO:teuthology.orchestra.run.smithi145.stdout:(2/8): libquadmath-8.4.1-1.el8.x86_64.rpm 4.5 MB/s | 169 kB 00:00 2023-01-24T18:49:44.210 INFO:teuthology.orchestra.run.smithi145.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 13 MB/s | 225 kB 00:00 2023-01-24T18:49:44.272 INFO:teuthology.orchestra.run.smithi145.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 61 MB/s | 3.7 MB 00:00 2023-01-24T18:49:44.345 INFO:teuthology.orchestra.run.smithi145.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 64 MB/s | 4.7 MB 00:00 2023-01-24T18:49:44.398 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:44.399 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-01-24T18:49:44.400 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:49:44.400 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-01-24T18:49:44.400 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:49:44.400 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T18:49:44.401 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:49:44.401 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T18:49:44.401 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T18:49:44.401 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T18:49:44.401 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T18:49:44.402 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T18:49:44.402 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T18:49:44.402 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T18:49:44.402 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:44.402 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:49:44.403 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-01-24T18:49:44.403 INFO:teuthology.orchestra.run.smithi167.stdout:Install 8 Packages 2023-01-24T18:49:44.403 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:44.404 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 36 M 2023-01-24T18:49:44.404 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 199 M 2023-01-24T18:49:44.404 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:49:44.443 INFO:teuthology.orchestra.run.smithi167.stdout:(1/8): libquadmath-8.4.1-1.el8.x86_64.rpm 4.0 MB/s | 169 kB 00:00 2023-01-24T18:49:44.455 INFO:teuthology.orchestra.run.smithi167.stdout:(2/8): libgfortran-8.4.1-1.el8.x86_64.rpm 12 MB/s | 639 kB 00:00 2023-01-24T18:49:44.461 INFO:teuthology.orchestra.run.smithi145.stdout:(6/8): python3-scipy-1.0.0-20.module+el8.1.0+33 54 MB/s | 15 MB 00:00 2023-01-24T18:49:44.475 INFO:teuthology.orchestra.run.smithi167.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 12 MB/s | 225 kB 00:00 2023-01-24T18:49:44.500 INFO:teuthology.orchestra.run.smithi145.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 30 MB/s | 4.6 MB 00:00 2023-01-24T18:49:44.540 INFO:teuthology.orchestra.run.smithi167.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 57 MB/s | 3.7 MB 00:00 2023-01-24T18:49:44.621 INFO:teuthology.orchestra.run.smithi167.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 59 MB/s | 4.7 MB 00:00 2023-01-24T18:49:44.731 INFO:teuthology.orchestra.run.smithi145.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 13 MB/s | 7.4 MB 00:00 2023-01-24T18:49:44.733 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:44.733 INFO:teuthology.orchestra.run.smithi145.stdout:Total 62 MB/s | 36 MB 00:00 2023-01-24T18:49:44.743 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:49:44.744 INFO:teuthology.orchestra.run.smithi167.stdout:(6/8): python3-scipy-1.0.0-20.module+el8.1.0+33 48 MB/s | 15 MB 00:00 2023-01-24T18:49:44.761 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:49:44.761 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:49:44.789 INFO:teuthology.orchestra.run.smithi167.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 27 MB/s | 4.6 MB 00:00 2023-01-24T18:49:45.144 INFO:teuthology.orchestra.run.smithi167.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 10 MB/s | 7.4 MB 00:00 2023-01-24T18:49:45.145 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:45.146 INFO:teuthology.orchestra.run.smithi167.stdout:Total 48 MB/s | 36 MB 00:00 2023-01-24T18:49:45.155 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:49:45.174 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:49:45.174 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:49:45.193 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:49:45.196 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:49:45.573 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:49:45.577 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:49:45.814 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:49:45.980 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:49:46.120 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:49:46.211 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:49:46.271 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:46.380 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:49:46.539 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:49:46.688 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:46.821 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:46.988 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:49:47.221 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:47.430 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:49:47.523 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:49:47.697 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:49:47.970 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:49:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:49:48.407 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:49:48.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:49:48.910 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:49:48.979 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:49.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:49:49.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:51.027 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:51.778 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:52.279 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T18:49:52.445 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:49:52.744 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:49:52.744 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T18:49:52.745 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:52.745 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 3/8 2023-01-24T18:49:52.745 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2023-01-24T18:49:52.745 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:49:52.746 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:52.746 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 7/8 2023-01-24T18:49:53.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T18:49:53.254 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:49:53.552 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:49:53.553 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T18:49:53.553 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:49:53.553 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 3/8 2023-01-24T18:49:53.553 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2023-01-24T18:49:53.553 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:49:53.554 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:49:53.554 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 7/8 2023-01-24T18:49:53.655 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 8/8 2023-01-24T18:49:53.656 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:14 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:54.470 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 8/8 2023-01-24T18:49:54.471 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:15 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:55.591 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:49:55.691 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:55.691 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:49:55.692 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:55.692 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T18:49:55.692 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T18:49:55.692 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:55.693 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:49:55.832 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-rook 2023-01-24T18:49:56.192 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:56.434 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:49:56.533 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:56.533 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:49:56.533 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:49:56.534 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T18:49:56.534 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T18:49:56.535 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T18:49:56.535 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T18:49:56.536 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T18:49:56.536 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T18:49:56.536 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T18:49:56.537 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:56.537 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:49:56.675 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-rook 2023-01-24T18:49:56.977 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:17 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:57.037 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:49:57.776 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:18 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:49:58.787 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:49:58.788 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================================= 2023-01-24T18:49:58.788 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:49:58.789 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================================= 2023-01-24T18:49:58.789 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:49:58.789 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T18:49:58.789 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:49:58.789 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T18:49:58.790 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T18:49:58.790 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T18:49:58.790 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T18:49:58.790 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T18:49:58.790 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T18:49:58.791 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T18:49:58.791 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T18:49:58.791 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T18:49:58.791 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:49:58.791 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T18:49:58.792 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T18:49:58.792 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:58.792 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:49:58.792 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================================= 2023-01-24T18:49:58.792 INFO:teuthology.orchestra.run.smithi145.stdout:Install 13 Packages 2023-01-24T18:49:58.793 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:49:58.793 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 1.9 M 2023-01-24T18:49:58.793 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 25 M 2023-01-24T18:49:58.794 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:49:58.873 INFO:teuthology.orchestra.run.smithi145.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 405 kB/s | 33 kB 00:00 2023-01-24T18:49:58.874 INFO:teuthology.orchestra.run.smithi145.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 197 kB/s | 16 kB 00:00 2023-01-24T18:49:58.926 INFO:teuthology.orchestra.run.smithi145.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.5 MB/s | 81 kB 00:00 2023-01-24T18:49:58.934 INFO:teuthology.orchestra.run.smithi145.stdout:(4/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 328 kB/s | 47 kB 00:00 2023-01-24T18:49:58.954 INFO:teuthology.orchestra.run.smithi145.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 2.2 MB/s | 61 kB 00:00 2023-01-24T18:49:58.997 INFO:teuthology.orchestra.run.smithi145.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.5 MB/s | 155 kB 00:00 2023-01-24T18:49:59.014 INFO:teuthology.orchestra.run.smithi145.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 2.5 MB/s | 43 kB 00:00 2023-01-24T18:49:59.030 INFO:teuthology.orchestra.run.smithi145.stdout:(8/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 1.6 MB/s | 27 kB 00:00 2023-01-24T18:49:59.042 INFO:teuthology.orchestra.run.smithi145.stdout:(9/13): python3-websocket-client-0.56.0-5.el8.n 571 kB/s | 61 kB 00:00 2023-01-24T18:49:59.048 INFO:teuthology.orchestra.run.smithi145.stdout:(10/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.3 MB/s | 126 kB 00:00 2023-01-24T18:49:59.068 INFO:teuthology.orchestra.run.smithi145.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 6.0 MB/s | 1.2 MB 00:00 2023-01-24T18:49:59.069 INFO:teuthology.orchestra.run.smithi145.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 1.0 MB/s | 20 kB 00:00 2023-01-24T18:49:59.079 INFO:teuthology.orchestra.run.smithi145.stdout:(13/13): python3-pyasn1-modules-0.3.7-6.el8.noa 2.9 MB/s | 110 kB 00:00 2023-01-24T18:49:59.079 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:59.080 INFO:teuthology.orchestra.run.smithi145.stdout:Total 6.7 MB/s | 1.9 MB 00:00 2023-01-24T18:49:59.085 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:49:59.099 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:49:59.099 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:49:59.185 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:49:59.190 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:49:59.539 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:49:59.541 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================================= 2023-01-24T18:49:59.541 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:49:59.541 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================================= 2023-01-24T18:49:59.542 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:49:59.542 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T18:49:59.542 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:49:59.542 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T18:49:59.542 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T18:49:59.543 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T18:49:59.543 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T18:49:59.543 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T18:49:59.543 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T18:49:59.543 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T18:49:59.544 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T18:49:59.544 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T18:49:59.544 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:49:59.544 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T18:49:59.544 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T18:49:59.545 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:59.545 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:49:59.545 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================================= 2023-01-24T18:49:59.546 INFO:teuthology.orchestra.run.smithi167.stdout:Install 13 Packages 2023-01-24T18:49:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:49:59.546 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 1.9 M 2023-01-24T18:49:59.547 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 25 M 2023-01-24T18:49:59.547 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:49:59.634 INFO:teuthology.orchestra.run.smithi167.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 369 kB/s | 33 kB 00:00 2023-01-24T18:49:59.634 INFO:teuthology.orchestra.run.smithi167.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 180 kB/s | 16 kB 00:00 2023-01-24T18:49:59.687 INFO:teuthology.orchestra.run.smithi167.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.5 MB/s | 81 kB 00:00 2023-01-24T18:49:59.697 INFO:teuthology.orchestra.run.smithi167.stdout:(4/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 307 kB/s | 47 kB 00:00 2023-01-24T18:49:59.697 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:49:59.714 INFO:teuthology.orchestra.run.smithi167.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 2.2 MB/s | 61 kB 00:00 2023-01-24T18:49:59.753 INFO:teuthology.orchestra.run.smithi167.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.8 MB/s | 155 kB 00:00 2023-01-24T18:49:59.767 INFO:teuthology.orchestra.run.smithi167.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 3.0 MB/s | 43 kB 00:00 2023-01-24T18:49:59.780 INFO:teuthology.orchestra.run.smithi167.stdout:(8/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 2.0 MB/s | 27 kB 00:00 2023-01-24T18:49:59.795 INFO:teuthology.orchestra.run.smithi167.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.6 MB/s | 126 kB 00:00 2023-01-24T18:49:59.804 INFO:teuthology.orchestra.run.smithi167.stdout:(10/13): python3-kubernetes-11.0.0-6.el8.noarch 6.9 MB/s | 1.2 MB 00:00 2023-01-24T18:49:59.806 INFO:teuthology.orchestra.run.smithi167.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 562 kB/s | 61 kB 00:00 2023-01-24T18:49:59.809 INFO:teuthology.orchestra.run.smithi167.stdout:(12/13): python3-pyasn1-modules-0.3.7-6.el8.noa 7.8 MB/s | 110 kB 00:00 2023-01-24T18:49:59.832 INFO:teuthology.orchestra.run.smithi167.stdout:(13/13): python3-jsonpointer-1.10-11.el8.noarch 714 kB/s | 20 kB 00:00 2023-01-24T18:49:59.833 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:49:59.833 INFO:teuthology.orchestra.run.smithi167.stdout:Total 6.6 MB/s | 1.9 MB 00:00 2023-01-24T18:49:59.838 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:49:59.852 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:49:59.852 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:49:59.923 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T18:49:59.940 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:49:59.944 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:00.182 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T18:50:00.409 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T18:50:00.456 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:00.627 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-24T18:50:00.740 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T18:50:00.901 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-24T18:50:01.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T18:50:01.202 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T18:50:01.210 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T18:50:01.444 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-24T18:50:01.484 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T18:50:01.692 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-24T18:50:01.695 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T18:50:01.889 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T18:50:01.976 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T18:50:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T18:50:02.249 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T18:50:02.477 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T18:50:02.694 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T18:50:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T18:50:03.321 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T18:50:03.612 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T18:50:03.727 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:50:04.023 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:50:04.023 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T18:50:04.023 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T18:50:04.023 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T18:50:04.024 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T18:50:04.024 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T18:50:04.024 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T18:50:04.024 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T18:50:04.025 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T18:50:04.025 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T18:50:04.025 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2023-01-24T18:50:04.025 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2023-01-24T18:50:04.025 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/13 2023-01-24T18:50:04.120 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T18:50:04.430 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T18:50:04.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:50:04.875 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:50:04.876 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T18:50:04.876 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T18:50:04.876 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T18:50:04.876 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T18:50:04.876 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T18:50:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T18:50:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T18:50:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T18:50:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T18:50:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2023-01-24T18:50:04.878 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2023-01-24T18:50:04.878 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/13 2023-01-24T18:50:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-24T18:50:05.046 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:26 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:05.869 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-24T18:50:05.869 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:26 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:07.118 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:07.216 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:07.217 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:07.217 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:07.217 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T18:50:07.217 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T18:50:07.218 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T18:50:07.218 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T18:50:07.218 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T18:50:07.218 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T18:50:07.219 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T18:50:07.220 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:07.220 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:07.355 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T18:50:07.718 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:07.907 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:08.004 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:08.004 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:08.005 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:08.005 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T18:50:08.006 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T18:50:08.006 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T18:50:08.006 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T18:50:08.007 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T18:50:08.007 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T18:50:08.007 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T18:50:08.008 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T18:50:08.008 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T18:50:08.008 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T18:50:08.009 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T18:50:08.009 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T18:50:08.010 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:08.010 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:08.149 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T18:50:08.494 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:29 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:08.514 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:09.361 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:30 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:10.251 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:10.252 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================== 2023-01-24T18:50:10.252 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:50:10.252 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================== 2023-01-24T18:50:10.252 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:50:10.252 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T18:50:10.253 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:50:10.253 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T18:50:10.253 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T18:50:10.253 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T18:50:10.253 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T18:50:10.254 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:10.254 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:50:10.254 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================== 2023-01-24T18:50:10.254 INFO:teuthology.orchestra.run.smithi145.stdout:Install 5 Packages 2023-01-24T18:50:10.254 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:10.255 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 5.9 M 2023-01-24T18:50:10.255 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 26 M 2023-01-24T18:50:10.256 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:50:10.364 INFO:teuthology.orchestra.run.smithi145.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 547 kB/s | 60 kB 00:00 2023-01-24T18:50:10.419 INFO:teuthology.orchestra.run.smithi145.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 695 kB/s | 115 kB 00:00 2023-01-24T18:50:10.455 INFO:teuthology.orchestra.run.smithi145.stdout:(3/5): python3-babel-2.5.1-5.el8.noarch.rpm 53 MB/s | 4.8 MB 00:00 2023-01-24T18:50:10.456 INFO:teuthology.orchestra.run.smithi145.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.3 MB/s | 476 kB 00:00 2023-01-24T18:50:10.460 INFO:teuthology.orchestra.run.smithi145.stdout:(5/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 13 MB/s | 538 kB 00:00 2023-01-24T18:50:10.461 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:10.462 INFO:teuthology.orchestra.run.smithi145.stdout:Total 28 MB/s | 5.9 MB 00:00 2023-01-24T18:50:10.465 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:50:10.475 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:50:10.475 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:50:10.566 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:50:10.570 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:50:11.165 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:11.166 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================== 2023-01-24T18:50:11.166 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:50:11.166 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================== 2023-01-24T18:50:11.166 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:50:11.167 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T18:50:11.167 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:50:11.167 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T18:50:11.167 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T18:50:11.167 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T18:50:11.168 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T18:50:11.168 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:11.168 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:50:11.168 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================== 2023-01-24T18:50:11.169 INFO:teuthology.orchestra.run.smithi167.stdout:Install 5 Packages 2023-01-24T18:50:11.169 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:11.169 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 5.9 M 2023-01-24T18:50:11.170 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 26 M 2023-01-24T18:50:11.170 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:50:11.279 INFO:teuthology.orchestra.run.smithi167.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 545 kB/s | 60 kB 00:00 2023-01-24T18:50:11.338 INFO:teuthology.orchestra.run.smithi167.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 677 kB/s | 115 kB 00:00 2023-01-24T18:50:11.366 INFO:teuthology.orchestra.run.smithi167.stdout:(3/5): python3-babel-2.5.1-5.el8.noarch.rpm 55 MB/s | 4.8 MB 00:00 2023-01-24T18:50:11.368 INFO:teuthology.orchestra.run.smithi167.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.3 MB/s | 476 kB 00:00 2023-01-24T18:50:11.372 INFO:teuthology.orchestra.run.smithi167.stdout:(5/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 16 MB/s | 538 kB 00:00 2023-01-24T18:50:11.373 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:11.374 INFO:teuthology.orchestra.run.smithi167.stdout:Total 29 MB/s | 5.9 MB 00:00 2023-01-24T18:50:11.377 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:50:11.387 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:50:11.387 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:50:11.493 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:50:11.497 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:11.662 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:50:12.031 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T18:50:12.254 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T18:50:12.526 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:12.614 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:50:12.863 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T18:50:12.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T18:50:13.058 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:50:13.156 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T18:50:13.326 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:50:13.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T18:50:13.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T18:50:13.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:50:13.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T18:50:13.517 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:50:13.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T18:50:13.968 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:50:14.230 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:50:14.230 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T18:50:14.230 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T18:50:14.231 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:50:14.231 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T18:50:14.274 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T18:50:14.275 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:15.141 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T18:50:15.141 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:36 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:16.314 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:16.414 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:16.415 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:16.415 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:16.415 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T18:50:16.415 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T18:50:16.415 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T18:50:16.416 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T18:50:16.416 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:16.416 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:16.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-fuse 2023-01-24T18:50:16.899 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:17.237 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:17.333 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:17.334 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:17.334 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:17.334 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T18:50:17.335 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T18:50:17.335 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T18:50:17.335 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T18:50:17.335 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:17.335 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:17.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-fuse 2023-01-24T18:50:17.760 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:38 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:17.880 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:18.656 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:39 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:19.515 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:19.515 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:19.516 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:50:19.516 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:19.516 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:50:19.516 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse 2023-01-24T18:50:19.517 INFO:teuthology.orchestra.run.smithi145.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T18:50:19.517 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2023-01-24T18:50:19.517 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T18:50:19.517 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:19.517 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:50:19.518 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:19.518 INFO:teuthology.orchestra.run.smithi145.stdout:Install 2 Packages 2023-01-24T18:50:19.518 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:19.518 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 954 k 2023-01-24T18:50:19.519 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 3.0 M 2023-01-24T18:50:19.519 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:50:19.546 INFO:teuthology.orchestra.run.smithi145.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.7 MB/s | 83 kB 00:00 2023-01-24T18:50:19.755 INFO:teuthology.orchestra.run.smithi145.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 872 kB 00:00 2023-01-24T18:50:19.755 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:19.756 INFO:teuthology.orchestra.run.smithi145.stdout:Total 3.9 MB/s | 954 kB 00:00 2023-01-24T18:50:19.759 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:50:19.778 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:50:19.778 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:50:19.861 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:50:19.865 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:50:20.031 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:50:20.316 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T18:50:20.487 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:20.488 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:20.489 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:50:20.489 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:20.489 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:50:20.489 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse 2023-01-24T18:50:20.489 INFO:teuthology.orchestra.run.smithi167.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T18:50:20.490 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-01-24T18:50:20.490 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T18:50:20.490 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:20.491 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:50:20.491 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:20.491 INFO:teuthology.orchestra.run.smithi167.stdout:Install 2 Packages 2023-01-24T18:50:20.491 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:20.491 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 954 k 2023-01-24T18:50:20.492 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 3.0 M 2023-01-24T18:50:20.492 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:50:20.507 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:50:20.524 INFO:teuthology.orchestra.run.smithi167.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.3 MB/s | 83 kB 00:00 2023-01-24T18:50:20.819 INFO:teuthology.orchestra.run.smithi167.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 2.6 MB/s | 872 kB 00:00 2023-01-24T18:50:20.820 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:20.821 INFO:teuthology.orchestra.run.smithi167.stdout:Total 2.8 MB/s | 954 kB 00:00 2023-01-24T18:50:20.822 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:50:20.844 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:50:20.844 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:50:20.882 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:50:20.882 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T18:50:20.926 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:50:20.930 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:21.656 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:22.545 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T18:50:22.545 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:43 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:23.302 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T18:50:23.518 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:50:23.915 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:50:23.915 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T18:50:24.583 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:24.679 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:24.679 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:24.680 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T18:50:24.680 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:24.680 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:24.790 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-volume 2023-01-24T18:50:24.840 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T18:50:24.840 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:45 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:25.158 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:26.136 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:47 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:27.190 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:27.286 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T18:50:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:27.287 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:27.425 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-volume 2023-01-24T18:50:27.800 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:27.902 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:27.903 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:27.903 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2023-01-24T18:50:27.904 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:27.904 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:50:27.904 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T18:50:27.905 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:27.905 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:50:27.905 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:27.905 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:50:27.905 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:27.906 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 263 k 2023-01-24T18:50:27.906 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 1.3 M 2023-01-24T18:50:27.906 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:50:28.096 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.3 MB/s | 263 kB 00:00 2023-01-24T18:50:28.097 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:28.097 INFO:teuthology.orchestra.run.smithi145.stdout:Total 1.3 MB/s | 263 kB 00:00 2023-01-24T18:50:28.097 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:50:28.107 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:50:28.108 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:50:28.128 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:50:28.132 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:50:28.310 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:50:28.589 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:28.697 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:49 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:28.973 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:29.911 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:29.911 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:50 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:30.496 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:30.497 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:30.497 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-01-24T18:50:30.497 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:30.498 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:50:30.498 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T18:50:30.498 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:30.498 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:50:30.498 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:30.499 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:50:30.499 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:30.499 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 263 k 2023-01-24T18:50:30.499 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 1.3 M 2023-01-24T18:50:30.499 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:50:30.687 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.4 MB/s | 263 kB 00:00 2023-01-24T18:50:30.688 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:30.689 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.3 MB/s | 263 kB 00:00 2023-01-24T18:50:30.689 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:50:30.699 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:50:30.699 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:50:30.725 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:50:30.729 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:30.915 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:31.207 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:31.649 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:32.003 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:32.098 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:32.098 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:32.098 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:32.099 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:32.099 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:32.219 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librados-devel 2023-01-24T18:50:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:32.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:50:32.607 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:53 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:33.486 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:54 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:34.659 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:34.755 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:34.755 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:34.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:50:34.756 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:34.756 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:34.876 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados-devel 2023-01-24T18:50:35.249 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:35.320 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:35.321 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:35.322 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2023-01-24T18:50:35.322 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:35.322 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:50:35.322 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T18:50:35.323 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:35.323 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:50:35.323 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:35.323 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:50:35.324 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:35.324 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 130 k 2023-01-24T18:50:35.324 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 464 k 2023-01-24T18:50:35.324 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:50:35.489 INFO:teuthology.orchestra.run.smithi145.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 779 kB/s | 130 kB 00:00 2023-01-24T18:50:35.490 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:35.491 INFO:teuthology.orchestra.run.smithi145.stdout:Total 773 kB/s | 130 kB 00:00 2023-01-24T18:50:35.491 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:50:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:50:35.497 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:50:35.552 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:50:35.557 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:50:35.690 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:50:35.887 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:36.074 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:02:57 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:36.154 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:37.081 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:37.082 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:02:58 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:37.865 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:37.866 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:37.866 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-01-24T18:50:37.867 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:37.867 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:50:37.867 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T18:50:37.867 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:37.867 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:50:37.868 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:37.868 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:50:37.868 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:37.868 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 130 k 2023-01-24T18:50:37.868 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 464 k 2023-01-24T18:50:37.869 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:50:38.033 INFO:teuthology.orchestra.run.smithi167.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 780 kB/s | 130 kB 00:00 2023-01-24T18:50:38.034 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:38.034 INFO:teuthology.orchestra.run.smithi167.stdout:Total 774 kB/s | 130 kB 00:00 2023-01-24T18:50:38.035 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:50:38.041 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:50:38.042 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:50:38.097 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:50:38.102 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:38.221 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:38.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:38.691 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:39.152 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:39.249 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:39.249 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:39.250 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:50:39.250 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:39.250 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:39.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install libcephfs2 2023-01-24T18:50:39.747 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:39.760 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:39.761 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:00 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:40.523 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:01 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:41.691 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:41.786 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:41.787 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:41.787 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:50:41.787 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:41.787 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:41.838 INFO:teuthology.orchestra.run.smithi145.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:41.919 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs2 2023-01-24T18:50:42.285 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:42.292 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:42.294 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:50:42.294 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:42.396 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install libcephfs-devel 2023-01-24T18:50:42.761 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:43.075 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:04 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:43.496 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:04 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:44.627 INFO:teuthology.orchestra.run.smithi167.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:45.082 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:45.083 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:50:45.083 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:45.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs-devel 2023-01-24T18:50:45.270 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:45.271 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:45.271 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2023-01-24T18:50:45.272 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:45.272 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:50:45.272 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T18:50:45.272 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:45.272 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:50:45.273 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:50:45.273 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:50:45.273 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:45.273 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 25 k 2023-01-24T18:50:45.273 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 109 k 2023-01-24T18:50:45.274 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:50:45.395 INFO:teuthology.orchestra.run.smithi145.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 201 kB/s | 25 kB 00:00 2023-01-24T18:50:45.396 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:45.396 INFO:teuthology.orchestra.run.smithi145.stdout:Total 199 kB/s | 25 kB 00:00 2023-01-24T18:50:45.397 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:50:45.400 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:50:45.400 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:50:45.402 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:50:45.406 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:50:45.456 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:50:45.558 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:45.636 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:45.872 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:46.326 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:07 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:46.741 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:46.741 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:07 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:48.095 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:48.097 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:48.097 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-01-24T18:50:48.097 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:48.097 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:50:48.098 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T18:50:48.098 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:48.098 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:50:48.098 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:50:48.099 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:50:48.099 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:48.099 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 25 k 2023-01-24T18:50:48.099 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 109 k 2023-01-24T18:50:48.100 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:50:48.218 INFO:teuthology.orchestra.run.smithi167.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 205 kB/s | 25 kB 00:00 2023-01-24T18:50:48.219 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:50:48.219 INFO:teuthology.orchestra.run.smithi167.stdout:Total 202 kB/s | 25 kB 00:00 2023-01-24T18:50:48.220 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:50:48.223 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:50:48.223 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:50:48.225 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:50:48.229 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:50:48.283 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:50:48.428 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:48.672 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:48.674 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:50:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:48.772 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:50:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:50:48.772 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:50:48.773 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:48.883 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librados2 2023-01-24T18:50:49.248 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:49.543 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:50:49.543 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:10 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:50.009 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:11 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:51.376 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:50:51.395 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:51.473 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:50:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:50:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:50:51.473 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:51.608 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados2 2023-01-24T18:50:51.845 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:51.847 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:50:51.847 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:51.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install librbd1 2023-01-24T18:50:51.973 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:52.302 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:52.752 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:13 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:53.231 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:14 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:54.079 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:54.531 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:54.532 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:50:54.532 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:54.536 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:54.637 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librbd1 2023-01-24T18:50:54.991 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:54.992 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:50:54.992 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:55.004 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:55.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rados 2023-01-24T18:50:55.457 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:55.779 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:16 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:56.223 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:17 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:57.153 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:57.535 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:50:57.610 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:50:57.612 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:50:57.612 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:50:57.720 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rados 2023-01-24T18:50:57.991 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:50:57.992 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:50:57.992 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:50:58.084 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rgw 2023-01-24T18:50:58.086 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:58.449 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:50:59.064 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:20 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:50:59.222 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:20 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:00.378 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:00.568 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:00.836 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:00.837 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:51:00.837 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:00.968 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rgw 2023-01-24T18:51:01.028 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:01.029 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:51:01.029 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:01.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-cephfs 2023-01-24T18:51:01.336 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:01.489 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:02.195 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:23 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:02.355 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:23 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:03.502 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:03.676 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:03.961 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:03.963 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:51:03.963 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:04.068 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-cephfs 2023-01-24T18:51:04.129 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:04.130 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:51:04.130 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:04.258 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install python3-rbd 2023-01-24T18:51:04.435 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:04.622 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:05.158 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:26 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:05.365 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:26 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:06.502 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:06.758 INFO:teuthology.orchestra.run.smithi145.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:06.956 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:06.957 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:51:06.957 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:07.067 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rbd 2023-01-24T18:51:07.205 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:07.206 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2023-01-24T18:51:07.206 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:07.312 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-fuse 2023-01-24T18:51:07.431 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:07.678 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:08.172 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:29 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:08.432 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:29 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:09.543 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:51:09.991 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:09.993 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-01-24T18:51:09.993 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:10.094 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-fuse 2023-01-24T18:51:10.267 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:10.268 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:10.269 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2023-01-24T18:51:10.269 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:10.269 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:51:10.269 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T18:51:10.270 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:10.270 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:51:10.270 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:10.270 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:51:10.270 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:10.271 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 90 k 2023-01-24T18:51:10.271 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 264 k 2023-01-24T18:51:10.271 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:51:10.414 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 623 kB/s | 90 kB 00:00 2023-01-24T18:51:10.415 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:10.415 INFO:teuthology.orchestra.run.smithi145.stdout:Total 616 kB/s | 90 kB 00:00 2023-01-24T18:51:10.416 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:51:10.427 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:51:10.427 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:51:10.459 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:10.482 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:51:10.486 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:51:10.594 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:51:10.780 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:11.028 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:11.217 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:32 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:11.974 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:11.975 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:32 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:13.071 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:13.072 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:13.073 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-01-24T18:51:13.073 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:13.073 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:51:13.073 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T18:51:13.074 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:13.074 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:51:13.074 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:13.074 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:51:13.075 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:13.075 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 90 k 2023-01-24T18:51:13.075 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 264 k 2023-01-24T18:51:13.075 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:51:13.216 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 631 kB/s | 90 kB 00:00 2023-01-24T18:51:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:13.217 INFO:teuthology.orchestra.run.smithi167.stdout:Total 625 kB/s | 90 kB 00:00 2023-01-24T18:51:13.218 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:51:13.228 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:51:13.229 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:51:13.283 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:51:13.288 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:51:13.398 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:51:13.600 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:13.869 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:13.921 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:51:14.016 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:14.017 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:51:14.017 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:14.017 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:14.017 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:14.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-mirror 2023-01-24T18:51:14.499 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:14.756 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:14.756 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:35 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:15.232 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:36 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:16.769 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:51:16.868 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:16.868 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:51:16.868 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:16.868 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:16.868 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:17.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-mirror 2023-01-24T18:51:17.073 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:17.074 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:17.074 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repo Size 2023-01-24T18:51:17.075 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:17.075 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:51:17.075 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:51:17.075 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:17.076 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:51:17.076 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:17.076 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:51:17.076 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:17.076 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 3.5 M 2023-01-24T18:51:17.077 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 16 M 2023-01-24T18:51:17.077 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:51:17.374 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:17.619 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 6.3 MB/s | 3.5 MB 00:00 2023-01-24T18:51:17.620 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:17.621 INFO:teuthology.orchestra.run.smithi145.stdout:Total 6.3 MB/s | 3.5 MB 00:00 2023-01-24T18:51:17.622 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:51:17.634 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:51:17.635 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:51:17.701 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:51:17.706 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:51:18.120 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:39 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:18.147 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:51:18.322 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:18.715 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:19.651 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:19.651 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:40 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:19.876 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:19.877 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:19.877 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-01-24T18:51:19.877 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:19.877 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:51:19.878 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:51:19.878 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:19.878 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:51:19.878 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:19.878 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:51:19.879 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:19.879 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 3.5 M 2023-01-24T18:51:19.879 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 16 M 2023-01-24T18:51:19.879 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:51:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 6.8 MB/s | 3.5 MB 00:00 2023-01-24T18:51:20.390 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:20.390 INFO:teuthology.orchestra.run.smithi167.stdout:Total 6.7 MB/s | 3.5 MB 00:00 2023-01-24T18:51:20.391 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:51:20.404 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:51:20.405 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:51:20.473 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:51:20.477 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:51:20.920 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:51:21.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:21.541 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:21.710 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:51:21.808 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:21.808 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:51:21.808 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:21.809 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:21.809 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:21.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install rbd-nbd 2023-01-24T18:51:22.288 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:22.503 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:22.503 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:43 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:23.103 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:44 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:24.699 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:51:24.796 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:24.797 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:51:24.797 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:24.797 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:24.797 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:24.915 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2023-01-24T18:51:24.916 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:24.916 INFO:teuthology.orchestra.run.smithi145.stdout: Package Architecture Version Repository Size 2023-01-24T18:51:24.916 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2023-01-24T18:51:24.917 INFO:teuthology.orchestra.run.smithi145.stdout:Install 1 Package 2023-01-24T18:51:24.918 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:24.918 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 172 k 2023-01-24T18:51:24.918 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 548 k 2023-01-24T18:51:24.918 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2023-01-24T18:51:24.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-nbd 2023-01-24T18:51:25.092 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 986 kB/s | 172 kB 00:00 2023-01-24T18:51:25.092 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:25.094 INFO:teuthology.orchestra.run.smithi145.stdout:Total 978 kB/s | 172 kB 00:00 2023-01-24T18:51:25.094 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2023-01-24T18:51:25.105 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2023-01-24T18:51:25.105 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2023-01-24T18:51:25.160 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2023-01-24T18:51:25.165 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2023-01-24T18:51:25.295 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-01-24T18:51:25.299 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2023-01-24T18:51:25.461 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:25.768 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:26.153 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:47 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:26.695 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:26.695 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:03:47 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:27.915 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-01-24T18:51:27.916 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:27.916 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-01-24T18:51:27.916 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:27.917 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-01-24T18:51:27.917 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T18:51:27.917 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:27.917 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-01-24T18:51:27.917 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-01-24T18:51:27.918 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-01-24T18:51:27.918 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:27.918 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 172 k 2023-01-24T18:51:27.918 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 548 k 2023-01-24T18:51:27.918 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-01-24T18:51:28.086 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2023-01-24T18:51:28.087 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:51:28.087 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.0 MB/s | 172 kB 00:00 2023-01-24T18:51:28.087 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-01-24T18:51:28.100 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-01-24T18:51:28.100 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-01-24T18:51:28.158 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-01-24T18:51:28.162 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-01-24T18:51:28.305 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-01-24T18:51:28.490 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:28.752 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:28.944 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2023-01-24T18:51:29.039 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:29.039 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2023-01-24T18:51:29.040 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:29.040 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:29.040 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2023-01-24T18:51:29.155 DEBUG:teuthology.parallel:result is None 2023-01-24T18:51:29.716 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:51:29.716 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:03:50 ago on Tue 24 Jan 2023 06:47:39 PM UTC. 2023-01-24T18:51:31.623 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-01-24T18:51:31.721 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:31.721 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-01-24T18:51:31.721 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:51:31.721 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:31.722 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-01-24T18:51:31.850 DEBUG:teuthology.parallel:result is None 2023-01-24T18:51:31.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2023-01-24T18:51:31.896 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2023-01-24T18:51:31.897 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:51:31.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:51:32.047 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T18:51:32.070 INFO:teuthology.orchestra.run.smithi145.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T18:51:32.070 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T18:51:32.071 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T18:51:32.071 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2023-01-24T18:51:32.111 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2023-01-24T18:51:32.113 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:51:32.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:51:32.261 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T18:51:32.281 INFO:teuthology.orchestra.run.smithi167.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T18:51:32.281 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T18:51:32.281 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T18:51:32.282 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T18:51:32.282 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:32.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T18:51:32.311 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:32.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T18:51:32.345 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T18:51:32.346 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:32.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T18:51:32.378 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T18:51:32.444 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:32.445 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T18:51:32.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T18:51:32.534 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T18:51:32.535 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:32.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T18:51:32.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T18:51:32.626 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:32.626 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T18:51:32.652 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T18:51:32.716 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T18:51:32.781 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T18:51:32.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2023-01-24T18:51:32.784 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2023-01-24T18:51:32.807 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T18:51:32.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T18:51:32.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T18:51:32.871 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T18:51:32.871 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T18:51:32.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T18:51:32.948 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T18:51:32.948 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T18:51:32.949 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135607', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135607', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}, {'machine': {'ram': 15000}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi145.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgTaFh/6b/8RAKwHz8RfCnQGB5WOUdECya1GkbHQWNxTansiR8PKNwfkeiDpFmpwp0i0eANT1/DFgcPY/bvLeNocGcObszm6DKMXB3R76sCVF5bihjs/LC7b7sA3TBCXrsolh0X5c4PxmpXbZnxekz+V0vpshYpne+wuTKDbiWr39Gl8jfXs44UV/oDEMFbX+3+npHMEVQUlFhCoCTeJpPk+D0eLMPlKwHOfVTYGTUThQ2Mdlc4zmi5XGh5kPWS4RyIcC/0NjhKTSiQeL7OkGl1G/4ccI5ztT5p29ZPFoJNtOFz7qc0LeRD3PG98XNgP6oo1IrO1yKmrZ1tdVOCRxSJ69GOciwNBe+cYVtLl1NVkPLUMg/sCvETH+ZCvtNwOf6MfrtYZDu2Qd/6OXsceEUu6kIqLPC5jMAbdfAfvPWt61Oqks4aRR2Ry1sB3u0zmu2KhjmurQ4N8+S4sFkNXAFTOrwqwLQWf8y6pH1x8NM7R/P3RcRoW+z2DCBxeuKQo8=', 'smithi167.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NJNFqlUBucOXzFl8K9PLwkCJiVbyzGxrpb3vFXa5SjZG9rTRSW0HyN+BgfVYvgb8nV0lhKoZ4CXZvVRBzuAoF/+8CV91bXg7zF0COIj5ESs7WWYcwETDwt0xZjLLY+CC9DTy1qzN5ZFAHAA46+cfHGZ1Sz3VBjxSXQ0l8ElfufPY59B96kJxmL8+FMwaHNYIL/ZQ1YU9nCmvhkDrUYv7D/0eZ6NIFNN+WntrSZxl+eKO/e7Z4VIzh3gHBS7KFMb69Bg8gh0wUToax/JfBV4A8JpbuHEqjF8uqttKhO9IovqScLaYxij4Ou6TTicBvgn75iYj+P4XSi+Vnzkhk/q28RWli7bLhJEF7lWKia1WxPmLgow27W656Cv/bxroxSdiM1w71fvJCb2xKd1pvilPPKQk/Wkn5Hgtr5UgOSWQSHlYcTE6ENEsj8ixj5Cf3K5zQg5r18sBb9Nx+pdIgkMyheqeJ2T5kiPcpi/oDO1lrJvdiAIFtjRYkuVRVH1P31E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 0', 'sudo ceph osd pool set cache min_write_recency_for_promote 0']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T18:51:32.950 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T18:51:32.976 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T18:51:32.990 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T18:51:33.030 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T18:51:33.054 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:33.055 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T18:51:33.105 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T18:51:33.105 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2023-01-24T18:51:33.159 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T18:51:33.159 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:33.159 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 206930 Links: 1 2023-01-24T18:51:33.159 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:33.159 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:33.160 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-01-24 18:51:18.492142995 +0000 2023-01-24T18:51:33.160 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-01-24 18:46:56.874164898 +0000 2023-01-24T18:51:33.160 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-01-24 18:46:56.874164898 +0000 2023-01-24T18:51:33.161 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2023-01-24T18:51:33.161 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T18:51:33.220 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-01-24T18:51:33.221 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-01-24T18:51:33.221 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000244029 s, 2.1 MB/s 2023-01-24T18:51:33.222 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T18:51:33.278 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2023-01-24T18:51:33.332 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T18:51:33.333 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:33.333 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 200678 Links: 1 2023-01-24T18:51:33.333 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:33.333 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:33.333 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-01-24 18:51:18.493142974 +0000 2023-01-24T18:51:33.334 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-01-24 18:46:57.375154256 +0000 2023-01-24T18:51:33.334 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-01-24 18:46:57.375154256 +0000 2023-01-24T18:51:33.334 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2023-01-24T18:51:33.334 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T18:51:33.394 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-01-24T18:51:33.394 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-01-24T18:51:33.395 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000235603 s, 2.2 MB/s 2023-01-24T18:51:33.396 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T18:51:33.451 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2023-01-24T18:51:33.505 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T18:51:33.505 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:33.505 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 207993 Links: 1 2023-01-24T18:51:33.505 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:33.506 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:33.506 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-01-24 18:51:18.493142974 +0000 2023-01-24T18:51:33.506 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-01-24 18:46:57.868143785 +0000 2023-01-24T18:51:33.506 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-01-24 18:46:57.868143785 +0000 2023-01-24T18:51:33.506 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2023-01-24T18:51:33.507 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T18:51:33.568 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-01-24T18:51:33.568 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-01-24T18:51:33.568 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000198995 s, 2.6 MB/s 2023-01-24T18:51:33.569 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T18:51:33.622 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2023-01-24T18:51:33.677 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T18:51:33.677 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:33.678 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 204151 Links: 1 2023-01-24T18:51:33.678 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:33.678 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:33.679 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2023-01-24 18:51:18.493142974 +0000 2023-01-24T18:51:33.679 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2023-01-24 18:46:58.354133463 +0000 2023-01-24T18:51:33.679 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2023-01-24 18:46:58.354133463 +0000 2023-01-24T18:51:33.679 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2023-01-24T18:51:33.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T18:51:33.738 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2023-01-24T18:51:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2023-01-24T18:51:33.739 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000175705 s, 2.9 MB/s 2023-01-24T18:51:33.740 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T18:51:33.794 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:33.795 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:33.795 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T18:51:33.812 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T18:51:33.812 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-01-24T18:51:33.869 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T18:51:33.869 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:33.870 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 196565 Links: 1 2023-01-24T18:51:33.870 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:33.870 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:33.870 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-01-24 18:51:21.308339837 +0000 2023-01-24T18:51:33.870 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-01-24 18:46:56.874596878 +0000 2023-01-24T18:51:33.871 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-01-24 18:46:56.874596878 +0000 2023-01-24T18:51:33.871 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-01-24T18:51:33.871 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T18:51:33.935 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-01-24T18:51:33.935 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-01-24T18:51:33.935 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00032904 s, 1.6 MB/s 2023-01-24T18:51:33.936 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T18:51:33.993 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-01-24T18:51:34.049 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T18:51:34.049 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:34.050 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 204936 Links: 1 2023-01-24T18:51:34.050 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:34.050 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:34.050 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-01-24 18:51:21.309339819 +0000 2023-01-24T18:51:34.050 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-01-24 18:46:57.342588583 +0000 2023-01-24T18:51:34.051 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-01-24 18:46:57.342588583 +0000 2023-01-24T18:51:34.051 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-01-24T18:51:34.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T18:51:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-01-24T18:51:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-01-24T18:51:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000229006 s, 2.2 MB/s 2023-01-24T18:51:34.115 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T18:51:34.171 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-01-24T18:51:34.226 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T18:51:34.227 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:34.227 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 203398 Links: 1 2023-01-24T18:51:34.227 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:34.227 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:34.228 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-01-24 18:51:21.309339819 +0000 2023-01-24T18:51:34.229 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-01-24 18:46:57.816580182 +0000 2023-01-24T18:51:34.229 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-01-24 18:46:57.816580182 +0000 2023-01-24T18:51:34.229 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-01-24T18:51:34.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T18:51:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-01-24T18:51:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-01-24T18:51:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000229499 s, 2.2 MB/s 2023-01-24T18:51:34.292 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T18:51:34.349 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-01-24T18:51:34.403 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T18:51:34.403 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:51:34.404 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 203489 Links: 1 2023-01-24T18:51:34.404 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:51:34.404 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:51:34.404 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-01-24 18:51:21.309339819 +0000 2023-01-24T18:51:34.404 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-01-24 18:46:58.293571728 +0000 2023-01-24T18:51:34.405 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-01-24 18:46:58.293571728 +0000 2023-01-24T18:51:34.405 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-01-24T18:51:34.405 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T18:51:34.466 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-01-24T18:51:34.466 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-01-24T18:51:34.466 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000212057 s, 2.4 MB/s 2023-01-24T18:51:34.467 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T18:51:34.523 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:34.523 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:51:34.523 INFO:tasks.ceph:Generating config... 2023-01-24T18:51:34.525 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T18:51:34.525 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T18:51:34.525 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-01-24T18:51:34.526 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-24T18:51:34.526 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T18:51:34.527 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T18:51:34.528 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T18:51:34.528 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T18:51:34.528 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T18:51:34.529 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T18:51:34.529 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T18:51:34.530 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T18:51:34.530 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T18:51:34.530 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T18:51:34.530 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2023-01-24T18:51:34.531 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-24T18:51:34.531 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T18:51:34.531 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T18:51:34.531 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T18:51:34.532 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-01-24T18:51:34.535 INFO:tasks.ceph:[osd] osd map cache size = 1 2023-01-24T18:51:34.535 INFO:tasks.ceph:[osd] osd max backfills = 6 2023-01-24T18:51:34.536 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-24T18:51:34.536 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T18:51:34.536 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T18:51:34.537 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T18:51:34.537 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T18:51:34.537 INFO:tasks.ceph:[osd] osd scrub during recovery = False 2023-01-24T18:51:34.537 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-24T18:51:34.538 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-24T18:51:34.538 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T18:51:34.538 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-24T18:51:34.538 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-24T18:51:34.539 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-24T18:51:34.539 INFO:tasks.ceph:[global] ms type = async 2023-01-24T18:51:34.539 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2023-01-24T18:51:34.539 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2023-01-24T18:51:34.540 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-24T18:51:34.540 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-01-24T18:51:34.540 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T18:51:34.540 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T18:51:34.541 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T18:51:34.541 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T18:51:34.541 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T18:51:34.541 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-24T18:51:34.542 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-24T18:51:34.542 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-24T18:51:34.542 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-24T18:51:34.543 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T18:51:34.543 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-24T18:51:34.543 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T18:51:34.544 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T18:51:34.597 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T18:51:34.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T18:51:34.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:51:34.705 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.145'), ('mon.c', '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]'), ('mon.b', '172.21.15.167')] 2023-01-24T18:51:34.706 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790],172.21.15.167', 'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T18:51:34.707 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:34.707 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:51:34.760 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.145 --addv c '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]' --add b 172.21.15.167 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:34.841 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:34.841 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: generated fsid f7057319-7755-43a0-abfd-bc543eb5d39e 2023-01-24T18:51:34.841 INFO:teuthology.orchestra.run.smithi145.stdout:setting min_mon_release = octopus 2023-01-24T18:51:34.841 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 0 2023-01-24T18:51:34.842 INFO:teuthology.orchestra.run.smithi145.stdout:fsid f7057319-7755-43a0-abfd-bc543eb5d39e 2023-01-24T18:51:34.842 INFO:teuthology.orchestra.run.smithi145.stdout:last_changed 2023-01-24T18:51:34.840256+0000 2023-01-24T18:51:34.842 INFO:teuthology.orchestra.run.smithi145.stdout:created 2023-01-24T18:51:34.840256+0000 2023-01-24T18:51:34.842 INFO:teuthology.orchestra.run.smithi145.stdout:min_mon_release 15 (octopus) 2023-01-24T18:51:34.842 INFO:teuthology.orchestra.run.smithi145.stdout:election_strategy: 1 2023-01-24T18:51:34.843 INFO:teuthology.orchestra.run.smithi145.stdout:0: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.a 2023-01-24T18:51:34.843 INFO:teuthology.orchestra.run.smithi145.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2023-01-24T18:51:34.843 INFO:teuthology.orchestra.run.smithi145.stdout:2: [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] mon.c 2023-01-24T18:51:34.843 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T18:51:34.844 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:51:34.898 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f7057319-7755-43a0-abfd-bc543eb5d39e... 2023-01-24T18:51:34.901 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T18:51:34.941 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T18:51:34.979 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2023-01-24T18:51:34.980 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2023-01-24T18:51:34.980 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T18:51:34.980 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2023-01-24T18:51:34.980 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.980 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2023-01-24T18:51:34.981 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.981 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2023-01-24T18:51:34.981 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.981 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2023-01-24T18:51:34.981 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2023-01-24T18:51:34.982 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.982 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2023-01-24T18:51:34.982 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2023-01-24T18:51:34.982 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.982 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T18:51:34.983 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2023-01-24T18:51:34.983 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.983 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T18:51:34.983 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T18:51:34.983 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.984 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2023-01-24T18:51:34.984 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.984 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2023-01-24T18:51:34.984 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2023-01-24T18:51:34.984 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2023-01-24T18:51:34.985 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2023-01-24T18:51:34.985 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2023-01-24T18:51:34.985 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2023-01-24T18:51:34.985 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2023-01-24T18:51:34.985 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T18:51:34.986 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T18:51:34.986 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2023-01-24T18:51:34.986 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.986 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T18:51:34.986 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.987 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2023-01-24T18:51:34.987 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.987 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2023-01-24T18:51:34.987 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.987 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2023-01-24T18:51:34.988 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2023-01-24T18:51:34.988 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2023-01-24T18:51:34.988 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790],172.21.15.167" 2023-01-24T18:51:34.988 INFO:teuthology.orchestra.run.smithi145.stdout: mon election default strategy = 1 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: ms type = async 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug inject dispatch delay duration = 0.1 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug inject dispatch delay probability = 0.1 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_min_size = 2 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_size = 2 2023-01-24T18:51:34.989 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = f7057319-7755-43a0-abfd-bc543eb5d39e 2023-01-24T18:51:34.990 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.990 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2023-01-24T18:51:34.990 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2023-01-24T18:51:34.990 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.990 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2023-01-24T18:51:34.991 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 120 2023-01-24T18:51:34.991 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.991 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2023-01-24T18:51:34.991 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2023-01-24T18:51:34.991 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.992 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2023-01-24T18:51:34.992 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2023-01-24T18:51:34.992 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2023-01-24T18:51:34.992 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.992 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2023-01-24T18:51:34.993 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2023-01-24T18:51:34.993 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.993 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2023-01-24T18:51:34.993 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.994 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2023-01-24T18:51:34.994 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.994 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2023-01-24T18:51:34.994 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.994 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2023-01-24T18:51:34.995 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2023-01-24T18:51:34.995 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:34.995 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2023-01-24T18:51:34.995 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2023-01-24T18:51:34.995 INFO:teuthology.orchestra.run.smithi145.stdout: debug monc = 20 2023-01-24T18:51:34.996 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2023-01-24T18:51:34.996 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore compression algorithm = snappy 2023-01-24T18:51:34.996 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore compression mode = aggressive 2023-01-24T18:51:34.996 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2023-01-24T18:51:34.996 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore zero block detection = True 2023-01-24T18:51:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 20 2023-01-24T18:51:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 20 2023-01-24T18:51:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2023-01-24T18:51:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2023-01-24T18:51:34.997 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 10 2023-01-24T18:51:34.998 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T18:51:34.998 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2023-01-24T18:51:34.998 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2023-01-24T18:51:34.998 INFO:teuthology.orchestra.run.smithi145.stdout: osd backoff on degraded = True 2023-01-24T18:51:34.998 INFO:teuthology.orchestra.run.smithi145.stdout: osd backoff on peering = True 2023-01-24T18:51:34.999 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify cached snaps = True 2023-01-24T18:51:34.999 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify missing on start = True 2023-01-24T18:51:34.999 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 0.95 2023-01-24T18:51:34.999 INFO:teuthology.orchestra.run.smithi145.stdout: osd inject bad map crc probability = 0.1 2023-01-24T18:51:34.999 INFO:teuthology.orchestra.run.smithi145.stdout: osd map cache size = 1 2023-01-24T18:51:35.000 INFO:teuthology.orchestra.run.smithi145.stdout: osd max backfills = 6 2023-01-24T18:51:35.000 INFO:teuthology.orchestra.run.smithi145.stdout: osd max markdown count = 1000 2023-01-24T18:51:35.000 INFO:teuthology.orchestra.run.smithi145.stdout: osd mclock override recovery settings = True 2023-01-24T18:51:35.000 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = bluestore 2023-01-24T18:51:35.000 INFO:teuthology.orchestra.run.smithi145.stdout: osd op queue = debug_random 2023-01-24T18:51:35.001 INFO:teuthology.orchestra.run.smithi145.stdout: osd op queue cut off = debug_random 2023-01-24T18:51:35.001 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub during recovery = False 2023-01-24T18:51:35.001 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub min interval = 60 2023-01-24T18:51:35.001 INFO:teuthology.orchestra.run.smithi145.stdout: osd shutdown pgref assert = True 2023-01-24T18:51:35.001 INFO:teuthology.orchestra.run.smithi145.stdout: osd_max_pg_log_entries = 3000 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout: osd_min_pg_log_entries = 3000 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2023-01-24T18:51:35.002 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2023-01-24T18:51:35.003 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2023-01-24T18:51:35.003 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:51:35.003 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:51:35.003 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2023-01-24T18:51:35.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.004 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2023-01-24T18:51:35.004 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2023-01-24T18:51:35.004 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2023-01-24T18:51:35.004 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2023-01-24T18:51:35.004 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2023-01-24T18:51:35.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2023-01-24T18:51:35.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2023-01-24T18:51:35.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:51:35.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2023-01-24T18:51:35.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2023-01-24T18:51:35.006 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:51:35.006 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.006 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T18:51:35.006 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T18:51:35.006 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T18:51:35.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.007 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T18:51:35.007 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T18:51:35.007 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T18:51:35.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.008 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2023-01-24T18:51:35.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2023-01-24T18:51:35.008 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2023-01-24T18:51:35.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon min osdmap epochs = 50 2023-01-24T18:51:35.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon osdmap full prune interval = 2 2023-01-24T18:51:35.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon osdmap full prune min = 15 2023-01-24T18:51:35.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon osdmap full prune txsize = 2 2023-01-24T18:51:35.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon scrub interval = 300 2023-01-24T18:51:35.009 INFO:teuthology.orchestra.run.smithi145.stdout: paxos service trim min = 10 2023-01-24T18:51:35.010 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:51:35.010 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2023-01-24T18:51:35.010 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2023-01-24T18:51:35.010 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2023-01-24T18:51:35.010 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2023-01-24T18:51:35.011 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T18:51:35.011 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T18:51:35.011 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2023-01-24T18:51:35.011 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2023-01-24T18:51:35.011 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2023-01-24T18:51:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2023-01-24T18:51:35.013 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2023-01-24T18:51:35.014 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T18:51:35.014 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2023-01-24T18:51:35.014 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.014 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2023-01-24T18:51:35.014 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.015 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2023-01-24T18:51:35.015 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.015 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2023-01-24T18:51:35.015 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2023-01-24T18:51:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2023-01-24T18:51:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2023-01-24T18:51:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T18:51:35.017 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2023-01-24T18:51:35.017 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.017 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T18:51:35.017 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T18:51:35.017 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.018 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2023-01-24T18:51:35.018 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.018 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2023-01-24T18:51:35.018 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2023-01-24T18:51:35.018 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2023-01-24T18:51:35.019 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2023-01-24T18:51:35.019 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2023-01-24T18:51:35.019 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2023-01-24T18:51:35.019 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2023-01-24T18:51:35.020 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T18:51:35.020 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T18:51:35.020 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2023-01-24T18:51:35.020 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.020 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.021 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2023-01-24T18:51:35.022 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2023-01-24T18:51:35.022 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2023-01-24T18:51:35.022 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790],172.21.15.167" 2023-01-24T18:51:35.022 INFO:teuthology.orchestra.run.smithi167.stdout: mon election default strategy = 1 2023-01-24T18:51:35.022 INFO:teuthology.orchestra.run.smithi167.stdout: ms type = async 2023-01-24T18:51:35.023 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug inject dispatch delay duration = 0.1 2023-01-24T18:51:35.023 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug inject dispatch delay probability = 0.1 2023-01-24T18:51:35.023 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_min_size = 2 2023-01-24T18:51:35.023 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_size = 2 2023-01-24T18:51:35.024 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = f7057319-7755-43a0-abfd-bc543eb5d39e 2023-01-24T18:51:35.024 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2023-01-24T18:51:35.024 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2023-01-24T18:51:35.024 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.025 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2023-01-24T18:51:35.025 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 120 2023-01-24T18:51:35.025 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.025 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2023-01-24T18:51:35.025 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.026 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2023-01-24T18:51:35.027 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2023-01-24T18:51:35.027 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.027 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2023-01-24T18:51:35.027 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.027 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2023-01-24T18:51:35.028 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.028 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2023-01-24T18:51:35.028 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.028 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2023-01-24T18:51:35.029 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2023-01-24T18:51:35.029 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.029 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2023-01-24T18:51:35.029 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2023-01-24T18:51:35.029 INFO:teuthology.orchestra.run.smithi167.stdout: debug monc = 20 2023-01-24T18:51:35.030 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2023-01-24T18:51:35.030 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore compression algorithm = snappy 2023-01-24T18:51:35.030 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore compression mode = aggressive 2023-01-24T18:51:35.030 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2023-01-24T18:51:35.030 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore zero block detection = True 2023-01-24T18:51:35.031 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2023-01-24T18:51:35.031 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2023-01-24T18:51:35.031 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-01-24T18:51:35.031 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2023-01-24T18:51:35.031 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2023-01-24T18:51:35.032 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T18:51:35.032 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2023-01-24T18:51:35.032 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2023-01-24T18:51:35.032 INFO:teuthology.orchestra.run.smithi167.stdout: osd backoff on degraded = True 2023-01-24T18:51:35.032 INFO:teuthology.orchestra.run.smithi167.stdout: osd backoff on peering = True 2023-01-24T18:51:35.033 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify cached snaps = True 2023-01-24T18:51:35.033 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify missing on start = True 2023-01-24T18:51:35.033 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2023-01-24T18:51:35.033 INFO:teuthology.orchestra.run.smithi167.stdout: osd inject bad map crc probability = 0.1 2023-01-24T18:51:35.033 INFO:teuthology.orchestra.run.smithi167.stdout: osd map cache size = 1 2023-01-24T18:51:35.034 INFO:teuthology.orchestra.run.smithi167.stdout: osd max backfills = 6 2023-01-24T18:51:35.034 INFO:teuthology.orchestra.run.smithi167.stdout: osd max markdown count = 1000 2023-01-24T18:51:35.034 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock override recovery settings = True 2023-01-24T18:51:35.034 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2023-01-24T18:51:35.034 INFO:teuthology.orchestra.run.smithi167.stdout: osd op queue = debug_random 2023-01-24T18:51:35.035 INFO:teuthology.orchestra.run.smithi167.stdout: osd op queue cut off = debug_random 2023-01-24T18:51:35.035 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub during recovery = False 2023-01-24T18:51:35.035 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub min interval = 60 2023-01-24T18:51:35.035 INFO:teuthology.orchestra.run.smithi167.stdout: osd shutdown pgref assert = True 2023-01-24T18:51:35.035 INFO:teuthology.orchestra.run.smithi167.stdout: osd_max_pg_log_entries = 3000 2023-01-24T18:51:35.036 INFO:teuthology.orchestra.run.smithi167.stdout: osd_min_pg_log_entries = 3000 2023-01-24T18:51:35.036 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2023-01-24T18:51:35.036 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-01-24T18:51:35.036 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2023-01-24T18:51:35.037 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-01-24T18:51:35.037 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-01-24T18:51:35.037 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:51:35.037 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:51:35.037 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2023-01-24T18:51:35.038 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2023-01-24T18:51:35.038 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-01-24T18:51:35.038 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-01-24T18:51:35.038 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2023-01-24T18:51:35.039 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-01-24T18:51:35.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2023-01-24T18:51:35.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2023-01-24T18:51:35.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:51:35.039 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2023-01-24T18:51:35.040 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2023-01-24T18:51:35.040 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:51:35.040 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.040 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T18:51:35.040 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T18:51:35.041 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T18:51:35.041 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.041 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T18:51:35.041 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T18:51:35.041 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T18:51:35.042 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.042 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2023-01-24T18:51:35.042 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2023-01-24T18:51:35.042 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.042 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2023-01-24T18:51:35.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon min osdmap epochs = 50 2023-01-24T18:51:35.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon osdmap full prune interval = 2 2023-01-24T18:51:35.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon osdmap full prune min = 15 2023-01-24T18:51:35.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon osdmap full prune txsize = 2 2023-01-24T18:51:35.043 INFO:teuthology.orchestra.run.smithi167.stdout: mon scrub interval = 300 2023-01-24T18:51:35.044 INFO:teuthology.orchestra.run.smithi167.stdout: paxos service trim min = 10 2023-01-24T18:51:35.044 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-01-24T18:51:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2023-01-24T18:51:35.044 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2023-01-24T18:51:35.044 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2023-01-24T18:51:35.045 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2023-01-24T18:51:35.045 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T18:51:35.045 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T18:51:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2023-01-24T18:51:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2023-01-24T18:51:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2023-01-24T18:51:35.046 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T18:51:35.046 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T18:51:35.103 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T18:51:35.103 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:35.103 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T18:51:35.118 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:35.118 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T18:51:35.171 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:51:35.172 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:35.172 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T18:51:35.172 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:51:35.244 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:51:35.244 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:35.298 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:51:35.299 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:35.299 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T18:51:35.300 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:51:35.335 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:51:35.336 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:35.391 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T18:51:35.391 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T18:51:35.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T18:51:35.438 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T18:51:35.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:51:35.532 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:51:35.535 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T18:51:35.535 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T18:51:35.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T18:51:35.573 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T18:51:35.585 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T18:51:35.623 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T18:51:35.635 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T18:51:35.636 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:51:35.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T18:51:35.660 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:35.660 INFO:tasks.ceph:role: osd.0 2023-01-24T18:51:35.661 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:51:35.661 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T18:51:35.732 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:35.732 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:35.733 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:35.733 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 2023-01-24T18:51:35.733 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:35.734 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:35.734 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:35.734 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:35.735 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:35.735 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:36.102 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2023-01-24T18:51:36.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2023-01-24T18:51:36.110 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T18:51:36.224 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T18:51:36.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T18:51:36.318 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:36.319 INFO:tasks.ceph:role: osd.1 2023-01-24T18:51:36.319 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:51:36.319 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T18:51:36.390 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:36.390 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:36.390 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:36.390 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 2023-01-24T18:51:36.391 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:36.391 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:36.391 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:36.391 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:36.391 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:36.392 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:36.753 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2023-01-24T18:51:36.754 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2023-01-24T18:51:36.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T18:51:36.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T18:51:36.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T18:51:36.930 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:36.931 INFO:tasks.ceph:role: osd.2 2023-01-24T18:51:36.931 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:51:36.931 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T18:51:37.001 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 2023-01-24T18:51:37.002 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:37.003 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:37.003 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:37.004 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:37.366 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2023-01-24T18:51:37.367 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2023-01-24T18:51:37.368 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T18:51:37.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T18:51:37.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T18:51:37.555 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:37.555 INFO:tasks.ceph:role: osd.3 2023-01-24T18:51:37.555 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:51:37.556 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T18:51:37.626 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:37.627 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:37.627 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:37.627 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 2023-01-24T18:51:37.627 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:37.627 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:37.628 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:37.628 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:37.628 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:37.628 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:37.997 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2023-01-24T18:51:37.999 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2023-01-24T18:51:37.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T18:51:38.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T18:51:38.131 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:38.211 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.209+0000 7ff020a8e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T18:51:38.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.211+0000 7ff020a8e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T18:51:38.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.211+0000 7ff020a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:51:38.213 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.211+0000 7ff020a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:51:38.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.211+0000 7ff020a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:51:38.219 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:38.217+0000 7ff020a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T18:51:41.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T18:51:41.143 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:41.227 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.225+0000 7fbb34f0f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T18:51:41.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.225+0000 7fbb34f0f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T18:51:41.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.225+0000 7fbb34f0f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:51:41.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.225+0000 7fbb34f0f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:51:41.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.225+0000 7fbb34f0f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:51:41.228 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:41.226+0000 7fbb34f0f540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T18:51:44.089 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T18:51:44.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:44.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.199+0000 7f26ba6dd540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T18:51:44.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.200+0000 7f26ba6dd540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T18:51:44.202 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.200+0000 7f26ba6dd540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:51:44.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.200+0000 7f26ba6dd540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:51:44.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.200+0000 7f26ba6dd540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:51:44.203 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:44.200+0000 7f26ba6dd540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T18:51:47.059 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T18:51:47.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:47.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.211+0000 7fc12a38b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T18:51:47.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.211+0000 7fc12a38b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T18:51:47.214 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.211+0000 7fc12a38b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:51:47.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.211+0000 7fc12a38b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:51:47.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.211+0000 7fc12a38b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:51:47.215 INFO:teuthology.orchestra.run.smithi145.stderr:2023-01-24T18:51:47.212+0000 7fc12a38b540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T18:51:50.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T18:51:50.097 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T18:51:50.126 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:50.126 INFO:tasks.ceph:role: osd.4 2023-01-24T18:51:50.127 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:51:50.127 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T18:51:50.201 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:50.201 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:50.201 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:50.202 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-01-24T18:51:50.202 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:50.202 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:50.202 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:50.202 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:50.203 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:50.203 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:50.566 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-01-24T18:51:50.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-01-24T18:51:50.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T18:51:50.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T18:51:50.729 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T18:51:50.796 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:50.796 INFO:tasks.ceph:role: osd.5 2023-01-24T18:51:50.796 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:51:50.797 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T18:51:50.869 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:50.869 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-01-24T18:51:50.870 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:50.870 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:50.870 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:50.870 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:50.871 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:51.233 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-01-24T18:51:51.235 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-01-24T18:51:51.235 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T18:51:51.322 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T18:51:51.351 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T18:51:51.416 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:51.416 INFO:tasks.ceph:role: osd.6 2023-01-24T18:51:51.417 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:51:51.417 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T18:51:51.488 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:51.489 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:51.489 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:51.489 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-01-24T18:51:51.489 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:51.490 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:51.490 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:51.490 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:51.490 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:51.490 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:51.857 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-01-24T18:51:51.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-01-24T18:51:51.859 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T18:51:51.952 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T18:51:51.982 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T18:51:52.046 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:51:52.047 INFO:tasks.ceph:role: osd.7 2023-01-24T18:51:52.047 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2023-01-24T18:51:52.047 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T18:51:52.118 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:51:52.118 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:51:52.118 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:51:52.118 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-01-24T18:51:52.119 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:51:52.119 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:51:52.119 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:51:52.119 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:51:52.119 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:51:52.120 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:51:52.487 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-01-24T18:51:52.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-01-24T18:51:52.493 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T18:51:52.592 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T18:51:52.620 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:52.700 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.699+0000 7f0b15b06540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T18:51:52.702 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.701+0000 7f0b15b06540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T18:51:52.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.701+0000 7f0b15b06540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:51:52.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.701+0000 7f0b15b06540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:51:52.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.701+0000 7f0b15b06540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:51:52.710 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:52.708+0000 7f0b15b06540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T18:51:55.560 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T18:51:55.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:55.717 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.714+0000 7f058b30b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T18:51:55.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.715+0000 7f058b30b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T18:51:55.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.715+0000 7f058b30b540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:51:55.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.715+0000 7f058b30b540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:51:55.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.715+0000 7f058b30b540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:51:55.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:55.715+0000 7f058b30b540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T18:51:58.569 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T18:51:58.597 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:51:58.683 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.680+0000 7fb51c622540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T18:51:58.683 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.680+0000 7fb51c622540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T18:51:58.683 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.681+0000 7fb51c622540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:51:58.684 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.681+0000 7fb51c622540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:51:58.684 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.681+0000 7fb51c622540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:51:58.684 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:51:58.681+0000 7fb51c622540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T18:52:01.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T18:52:01.616 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:52:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T18:52:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T18:52:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:52:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:52:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:52:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2023-01-24T18:52:01.700+0000 7f2f943db540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T18:52:04.557 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T18:52:04.584 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T18:52:04.584 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:04.584 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T18:52:04.614 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:04.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T18:52:04.682 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:04.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T18:52:04.748 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:04.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T18:52:04.813 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:04.814 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T18:52:04.878 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:04.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T18:52:04.907 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:04.907 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T18:52:04.974 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:04.974 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T18:52:05.039 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:05.040 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T18:52:05.105 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:05.106 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T18:52:05.171 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:05.171 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T18:52:05.187 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:05.187 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T18:52:05.226 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T18:52:05.227 DEBUG:teuthology.orchestra.run.smithi145:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T18:52:05.229 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T18:52:05.270 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.y] 2023-01-24T18:52:05.270 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC3KNBjVhsSGhAAAbO2hyMv3iApON6ghEIOPg== 2023-01-24T18:52:05.270 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.0] 2023-01-24T18:52:05.270 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC6KNBjQLCSDBAAeuIBKiqFE7ghNAEkjWGSbg== 2023-01-24T18:52:05.271 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.1] 2023-01-24T18:52:05.271 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC9KNBjlWSADRAA7YU3AEfRqHTqm3497qA8lA== 2023-01-24T18:52:05.271 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.2] 2023-01-24T18:52:05.271 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDAKNBjEdD7CxAAvDHupAsDPK4Ce7UG7Z/QQQ== 2023-01-24T18:52:05.272 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.3] 2023-01-24T18:52:05.272 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDDKNBjXQKuDBAAo8HilfuJkl/kvhuzzSA34w== 2023-01-24T18:52:05.272 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.x] 2023-01-24T18:52:05.272 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC3KNBjXAyzHxAAYx2snTpDAE+cpXQGX2YUAw== 2023-01-24T18:52:05.272 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.4] 2023-01-24T18:52:05.273 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDIKNBjDqm9KRAA5meo3BlqVhWBtWjvgLkGDw== 2023-01-24T18:52:05.273 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.5] 2023-01-24T18:52:05.273 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDLKNBjnE+oKhAAMMZHvQWXfVCBFpPkpo7Uxg== 2023-01-24T18:52:05.273 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.6] 2023-01-24T18:52:05.273 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDOKNBjwaqgKBAAGqjLrPcAacf26HZVuxY+GA== 2023-01-24T18:52:05.274 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.7] 2023-01-24T18:52:05.274 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDRKNBjEFzKKRAAaG+FIBzWW/ess7MYWy2q2g== 2023-01-24T18:52:05.274 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2023-01-24T18:52:05.274 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC3KNBjJiEcIhAA0jm/QWfPhZlIL+ZnYEe8og== 2023-01-24T18:52:05.274 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2023-01-24T18:52:05.275 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQC3KNBjLvUbJRAAbmr87s5RAwYF7uk4MmN7NQ== 2023-01-24T18:52:05.289 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2023-01-24T18:52:05.290 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC3KNBjVhsSGhAAAbO2hyMv3iApON6ghEIOPg== 2023-01-24T18:52:05.290 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2023-01-24T18:52:05.290 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC6KNBjQLCSDBAAeuIBKiqFE7ghNAEkjWGSbg== 2023-01-24T18:52:05.291 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2023-01-24T18:52:05.291 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC9KNBjlWSADRAA7YU3AEfRqHTqm3497qA8lA== 2023-01-24T18:52:05.291 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2023-01-24T18:52:05.291 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDAKNBjEdD7CxAAvDHupAsDPK4Ce7UG7Z/QQQ== 2023-01-24T18:52:05.291 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2023-01-24T18:52:05.292 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDDKNBjXQKuDBAAo8HilfuJkl/kvhuzzSA34w== 2023-01-24T18:52:05.292 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2023-01-24T18:52:05.292 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC3KNBjXAyzHxAAYx2snTpDAE+cpXQGX2YUAw== 2023-01-24T18:52:05.293 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2023-01-24T18:52:05.293 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDIKNBjDqm9KRAA5meo3BlqVhWBtWjvgLkGDw== 2023-01-24T18:52:05.293 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2023-01-24T18:52:05.293 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDLKNBjnE+oKhAAMMZHvQWXfVCBFpPkpo7Uxg== 2023-01-24T18:52:05.293 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2023-01-24T18:52:05.294 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDOKNBjwaqgKBAAGqjLrPcAacf26HZVuxY+GA== 2023-01-24T18:52:05.294 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2023-01-24T18:52:05.294 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDRKNBjEFzKKRAAaG+FIBzWW/ess7MYWy2q2g== 2023-01-24T18:52:05.294 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2023-01-24T18:52:05.294 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC3KNBjJiEcIhAA0jm/QWfPhZlIL+ZnYEe8og== 2023-01-24T18:52:05.295 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2023-01-24T18:52:05.295 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC3KNBjLvUbJRAAbmr87s5RAwYF7uk4MmN7NQ== 2023-01-24T18:52:05.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:52:05.318 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:52:05.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.387 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.435 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.483 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:52:05.579 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:52:05.625 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.676 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.722 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.770 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.810 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:52:05.856 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:52:05.859 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:52:05.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:52:05.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:52:05.961 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T18:52:05.962 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T18:52:05.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:52:06.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T18:52:06.099 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T18:52:06.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:52:06.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T18:52:06.275 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T18:52:06.299 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:52:06.389 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T18:52:06.412 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:52:06.415 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:52:06.466 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T18:52:06.467 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T18:52:06.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T18:52:06.470 INFO:tasks.ceph.mon.a:Started 2023-01-24T18:52:06.470 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T18:52:06.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T18:52:06.472 INFO:tasks.ceph.mon.c:Started 2023-01-24T18:52:06.473 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T18:52:06.473 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T18:52:06.509 INFO:tasks.ceph.mon.b:Started 2023-01-24T18:52:06.509 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T18:52:06.509 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T18:52:06.511 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T18:52:06.514 INFO:tasks.ceph.mgr.y:Started 2023-01-24T18:52:06.514 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T18:52:06.514 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T18:52:06.516 INFO:tasks.ceph.mgr.x:Started 2023-01-24T18:52:06.517 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T18:52:06.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T18:52:11.899 INFO:teuthology.orchestra.run.smithi145.stderr:adjusted tunables profile to default 2023-01-24T18:52:11.910 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T18:52:11.911 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:11.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T18:52:11.935 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:11.936 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T18:52:11.999 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:12.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T18:52:12.062 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2023-01-24T18:52:12.063 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T18:52:12.125 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:12.126 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T18:52:12.151 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:12.152 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T18:52:12.215 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:12.216 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T18:52:12.279 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-01-24T18:52:12.280 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T18:52:12.343 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 20f62281-4976-49fd-9fcc-50fdeb27053c 0 2023-01-24T18:52:12.591 INFO:teuthology.orchestra.run.smithi167.stdout:0 2023-01-24T18:52:12.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new b78a83ab-291c-40d9-8b6e-d65ca043d34a 1 2023-01-24T18:52:12.819 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-01-24T18:52:12.832 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new e10a1379-65a2-40f5-9308-9385c268cbbb 2 2023-01-24T18:52:12.991 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:12.990+0000 7fa4375be180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T18:52:12.992 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:12.990+0000 7fa4375be180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T18:52:12.992 INFO:tasks.ceph.mgr.x.smithi167.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T18:52:12.993 INFO:tasks.ceph.mgr.x.smithi167.stderr: from .module import Module, StandbyModule 2023-01-24T18:52:12.993 INFO:tasks.ceph.mgr.x.smithi167.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T18:52:12.993 INFO:tasks.ceph.mgr.x.smithi167.stderr: from packaging.version import Version 2023-01-24T18:52:12.993 INFO:tasks.ceph.mgr.x.smithi167.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T18:52:12.993 INFO:tasks.ceph.mgr.x.smithi167.stderr: 2023-01-24T18:52:12.994 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:12.990+0000 7fa4375be180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T18:52:12.994 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:12.990+0000 7fa4375be180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T18:52:13.044 INFO:teuthology.orchestra.run.smithi167.stdout:2 2023-01-24T18:52:13.057 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 294a4836-c07e-4229-9d35-ef1211b721a7 3 2023-01-24T18:52:13.129 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.127+0000 7fa4375be180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T18:52:13.207 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.205+0000 7fa4375be180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T18:52:13.267 INFO:teuthology.orchestra.run.smithi167.stdout:3 2023-01-24T18:52:13.281 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 43f5962f-28f4-4970-a0e0-95f9ab3a1d47 4 2023-01-24T18:52:13.399 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.398+0000 7fa4375be180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:52:13.490 INFO:teuthology.orchestra.run.smithi167.stdout:4 2023-01-24T18:52:13.503 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new dbfc9c39-2a9a-496f-b135-c7456357985d 5 2023-01-24T18:52:13.579 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.577+0000 7fa4375be180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T18:52:13.660 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.658+0000 7fa4375be180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T18:52:13.689 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.687+0000 7f26146e3180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T18:52:13.690 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.687+0000 7f26146e3180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T18:52:13.690 INFO:tasks.ceph.mgr.y.smithi145.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T18:52:13.690 INFO:tasks.ceph.mgr.y.smithi145.stderr: from .module import Module, StandbyModule 2023-01-24T18:52:13.690 INFO:tasks.ceph.mgr.y.smithi145.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T18:52:13.691 INFO:tasks.ceph.mgr.y.smithi145.stderr: from packaging.version import Version 2023-01-24T18:52:13.691 INFO:tasks.ceph.mgr.y.smithi145.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T18:52:13.691 INFO:tasks.ceph.mgr.y.smithi145.stderr: 2023-01-24T18:52:13.691 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.688+0000 7f26146e3180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T18:52:13.691 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.688+0000 7f26146e3180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T18:52:13.712 INFO:teuthology.orchestra.run.smithi167.stdout:5 2023-01-24T18:52:13.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 8d0a58d6-f8e1-4728-922b-a360c114f939 6 2023-01-24T18:52:13.802 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.800+0000 7fa4375be180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T18:52:13.820 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.818+0000 7f26146e3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T18:52:13.894 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:13.892+0000 7f26146e3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T18:52:13.900 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:13.899+0000 7fa4375be180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T18:52:13.936 INFO:teuthology.orchestra.run.smithi167.stdout:6 2023-01-24T18:52:13.949 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 3a05c01e-9ac9-4e3f-a173-7dc4997671e0 7 2023-01-24T18:52:14.076 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:14.074+0000 7f26146e3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:52:14.155 INFO:teuthology.orchestra.run.smithi167.stdout:7 2023-01-24T18:52:14.168 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T18:52:14.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T18:52:14.170 INFO:tasks.ceph.osd.0:Started 2023-01-24T18:52:14.171 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T18:52:14.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T18:52:14.174 INFO:tasks.ceph.osd.1:Started 2023-01-24T18:52:14.174 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T18:52:14.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T18:52:14.176 INFO:tasks.ceph.osd.2:Started 2023-01-24T18:52:14.177 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T18:52:14.177 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T18:52:14.180 INFO:tasks.ceph.osd.3:Started 2023-01-24T18:52:14.180 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T18:52:14.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T18:52:14.182 INFO:tasks.ceph.osd.4:Started 2023-01-24T18:52:14.183 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T18:52:14.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T18:52:14.185 INFO:tasks.ceph.osd.5:Started 2023-01-24T18:52:14.185 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T18:52:14.186 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T18:52:14.188 INFO:tasks.ceph.osd.6:Started 2023-01-24T18:52:14.188 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T18:52:14.188 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T18:52:14.190 INFO:tasks.ceph.osd.7:Started 2023-01-24T18:52:14.191 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:52:14.285 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:14.283+0000 7f26146e3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T18:52:14.366 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:14.364+0000 7f26146e3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T18:52:14.401 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:14.401 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":10,"fsid":"f7057319-7755-43a0-abfd-bc543eb5d39e","created":"2023-01-24T18:52:11.784905+0000","modified":"2023-01-24T18:52:14.150744+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:52:14.150744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"20f62281-4976-49fd-9fcc-50fdeb27053c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78a83ab-291c-40d9-8b6e-d65ca043d34a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e10a1379-65a2-40f5-9308-9385c268cbbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"294a4836-c07e-4229-9d35-ef1211b721a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"43f5962f-28f4-4970-a0e0-95f9ab3a1d47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dbfc9c39-2a9a-496f-b135-c7456357985d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8d0a58d6-f8e1-4728-922b-a360c114f939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3a05c01e-9ac9-4e3f-a173-7dc4997671e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:52:14.414 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T18:52:14.415 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T18:52:14.455 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:14.453+0000 7fa4375be180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T18:52:14.504 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:14.502+0000 7f26146e3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T18:52:14.530 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:14.529+0000 7fa4375be180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T18:52:14.598 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:14.596+0000 7f26146e3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T18:52:14.608 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:14.606+0000 7fa4375be180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T18:52:14.774 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:14.772+0000 7faf7eb1e540 -1 Falling back to public interface 2023-01-24T18:52:14.777 INFO:tasks.ceph.osd.0.smithi145.stderr:2023-01-24T18:52:14.775+0000 7f5f6bda3540 -1 Falling back to public interface 2023-01-24T18:52:14.785 INFO:tasks.ceph.osd.2.smithi145.stderr:2023-01-24T18:52:14.783+0000 7f2f87376540 -1 Falling back to public interface 2023-01-24T18:52:14.788 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-01-24T18:52:14.786+0000 7fd4f4652540 -1 Falling back to public interface 2023-01-24T18:52:14.789 INFO:tasks.ceph.osd.3.smithi145.stderr:2023-01-24T18:52:14.787+0000 7f9966585540 -1 Falling back to public interface 2023-01-24T18:52:14.793 INFO:tasks.ceph.osd.1.smithi145.stderr:2023-01-24T18:52:14.791+0000 7fc627b60540 -1 Falling back to public interface 2023-01-24T18:52:14.810 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-01-24T18:52:14.808+0000 7fceb5d97540 -1 Falling back to public interface 2023-01-24T18:52:14.819 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:14.817+0000 7f7f3b0c4540 -1 Falling back to public interface 2023-01-24T18:52:15.118 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:15.116+0000 7f26146e3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T18:52:15.193 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:15.191+0000 7f26146e3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T18:52:15.197 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:15.195+0000 7fa4375be180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T18:52:15.221 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:52:15.270 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:15.268+0000 7f26146e3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T18:52:15.333 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:15.331+0000 7fa4375be180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:52:15.403 INFO:teuthology.misc.health.smithi145.stdout: 2023-01-24T18:52:15.404 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":10,"fsid":"f7057319-7755-43a0-abfd-bc543eb5d39e","created":"2023-01-24T18:52:11.784905+0000","modified":"2023-01-24T18:52:14.150744+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:52:14.150744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"20f62281-4976-49fd-9fcc-50fdeb27053c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78a83ab-291c-40d9-8b6e-d65ca043d34a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e10a1379-65a2-40f5-9308-9385c268cbbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"294a4836-c07e-4229-9d35-ef1211b721a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"43f5962f-28f4-4970-a0e0-95f9ab3a1d47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dbfc9c39-2a9a-496f-b135-c7456357985d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8d0a58d6-f8e1-4728-922b-a360c114f939","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3a05c01e-9ac9-4e3f-a173-7dc4997671e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:52:15.416 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T18:52:15.431 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:15.429+0000 7fa4375be180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T18:52:15.678 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:15.676+0000 7fa4375be180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T18:52:15.754 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:15.752+0000 7fa4375be180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T18:52:15.843 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:15.841+0000 7f26146e3180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T18:52:15.979 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:15.977+0000 7f26146e3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:52:16.066 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:16.064+0000 7f26146e3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T18:52:16.312 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:16.310+0000 7f26146e3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T18:52:16.324 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:16.322+0000 7fa4375be180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T18:52:16.387 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:16.385+0000 7f26146e3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T18:52:16.566 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:16.565+0000 7fa4375be180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T18:52:16.697 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:16.696+0000 7fa4375be180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T18:52:16.783 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:16.781+0000 7fa4375be180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T18:52:16.929 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:16.927+0000 7fa4375be180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T18:52:16.965 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:16.963+0000 7f26146e3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T18:52:17.004 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:17.003+0000 7fa4375be180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T18:52:17.083 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:17.081+0000 7fa4375be180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T18:52:17.156 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:17.155+0000 7fa4375be180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T18:52:17.157 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:17.155+0000 7fa4375be180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T18:52:17.210 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.208+0000 7f26146e3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T18:52:17.352 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.350+0000 7f26146e3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T18:52:17.433 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.431+0000 7f26146e3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T18:52:17.653 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.651+0000 7f26146e3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T18:52:17.739 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.737+0000 7f26146e3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T18:52:17.813 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.811+0000 7f26146e3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T18:52:17.888 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.886+0000 7f26146e3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T18:52:17.888 INFO:tasks.ceph.mgr.y.smithi145.stderr:2023-01-24T18:52:17.886+0000 7f26146e3180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T18:52:18.060 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:18.059+0000 7faf7eb1e540 -1 osd.5 0 log_to_monitors true 2023-01-24T18:52:18.090 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-01-24T18:52:18.089+0000 7fd4f4652540 -1 osd.4 0 log_to_monitors true 2023-01-24T18:52:18.180 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:18.179+0000 7fa3f0a88700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T18:52:18.181 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-01-24T18:52:18.179+0000 7fa3f0a88700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T18:52:18.285 INFO:tasks.ceph.osd.0.smithi145.stderr:2023-01-24T18:52:18.283+0000 7f5f6bda3540 -1 osd.0 0 log_to_monitors true 2023-01-24T18:52:18.296 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-01-24T18:52:18.295+0000 7fceb5d97540 -1 osd.6 0 log_to_monitors true 2023-01-24T18:52:18.318 INFO:tasks.ceph.osd.1.smithi145.stderr:2023-01-24T18:52:18.316+0000 7fc627b60540 -1 osd.1 0 log_to_monitors true 2023-01-24T18:52:18.326 INFO:tasks.ceph.osd.2.smithi145.stderr:2023-01-24T18:52:18.325+0000 7f2f87376540 -1 osd.2 0 log_to_monitors true 2023-01-24T18:52:18.327 INFO:tasks.ceph.osd.3.smithi145.stderr:2023-01-24T18:52:18.325+0000 7f9966585540 -1 osd.3 0 log_to_monitors true 2023-01-24T18:52:18.382 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:18.381+0000 7f7f3b0c4540 -1 osd.7 0 log_to_monitors true 2023-01-24T18:52:19.566 INFO:tasks.ceph.osd.0.smithi145.stderr:2023-01-24T18:52:19.564+0000 7f5f641c7700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T18:52:19.567 INFO:tasks.ceph.osd.1.smithi145.stderr:2023-01-24T18:52:19.565+0000 7fc61e781700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T18:52:19.568 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-01-24T18:52:19.565+0000 7fd4eb273700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T18:52:19.568 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:19.565+0000 7faf7573f700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T18:52:19.569 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-01-24T18:52:19.566+0000 7fceac9b8700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T18:52:19.569 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:19.566+0000 7f7f334e8700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T18:52:19.570 INFO:tasks.ceph.osd.3.smithi145.stderr:2023-01-24T18:52:19.566+0000 7f995d1a6700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T18:52:19.570 INFO:tasks.ceph.osd.2.smithi145.stderr:2023-01-24T18:52:19.566+0000 7f2f7df97700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T18:52:19.570 INFO:tasks.ceph.osd.1.smithi145.stderr:2023-01-24T18:52:19.567+0000 7fc612f17700 -1 osd.1 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.571 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-01-24T18:52:19.566+0000 7fd4dfa09700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.571 INFO:tasks.ceph.osd.0.smithi145.stderr:2023-01-24T18:52:19.566+0000 7f5f5715a700 -1 osd.0 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.572 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-01-24T18:52:19.567+0000 7fcea114e700 -1 osd.6 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.573 INFO:tasks.ceph.osd.2.smithi145.stderr:2023-01-24T18:52:19.571+0000 7f2f7272d700 -1 osd.2 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.575 INFO:tasks.ceph.osd.3.smithi145.stderr:2023-01-24T18:52:19.573+0000 7f995193c700 -1 osd.3 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.576 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-01-24T18:52:19.574+0000 7fd4e486a700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.577 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:19.572+0000 7faf696d4700 -1 osd.5 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.577 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:19.573+0000 7f7f26c7c700 -1 osd.7 0 handle_osd_map injecting map crc failure 2023-01-24T18:52:19.578 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-01-24T18:52:19.576+0000 7fcea5faf700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.579 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:19.578+0000 7faf6e535700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.580 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:19.579+0000 7f7f2badd700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.581 INFO:tasks.ceph.osd.0.smithi145.stderr:2023-01-24T18:52:19.578+0000 7f5f5bfbb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.581 INFO:tasks.ceph.osd.1.smithi145.stderr:2023-01-24T18:52:19.579+0000 7fc617d78700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.583 INFO:tasks.ceph.osd.3.smithi145.stderr:2023-01-24T18:52:19.581+0000 7f995679d700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:19.583 INFO:tasks.ceph.osd.2.smithi145.stderr:2023-01-24T18:52:19.581+0000 7f2f7758e700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:52:22.222 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:52:22.527 INFO:teuthology.misc.health.smithi145.stdout: 2023-01-24T18:52:22.528 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":14,"fsid":"f7057319-7755-43a0-abfd-bc543eb5d39e","created":"2023-01-24T18:52:11.784905+0000","modified":"2023-01-24T18:52:21.583855+0000","last_up_change":"2023-01-24T18:52:20.579539+0000","last_in_change":"2023-01-24T18:52:14.150744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:52:21.206164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"20f62281-4976-49fd-9fcc-50fdeb27053c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":82439},{"type":"v1","addr":"172.21.15.145:6801","nonce":82439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":82439},{"type":"v1","addr":"172.21.15.145:6803","nonce":82439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":82439},{"type":"v1","addr":"172.21.15.145:6807","nonce":82439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":82439},{"type":"v1","addr":"172.21.15.145:6805","nonce":82439}]},"public_addr":"172.21.15.145:6801/82439","cluster_addr":"172.21.15.145:6803/82439","heartbeat_back_addr":"172.21.15.145:6807/82439","heartbeat_front_addr":"172.21.15.145:6805/82439","state":["exists","up"]},{"osd":1,"uuid":"b78a83ab-291c-40d9-8b6e-d65ca043d34a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":82446},{"type":"v1","addr":"172.21.15.145:6825","nonce":82446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":82446},{"type":"v1","addr":"172.21.15.145:6827","nonce":82446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":82446},{"type":"v1","addr":"172.21.15.145:6831","nonce":82446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":82446},{"type":"v1","addr":"172.21.15.145:6829","nonce":82446}]},"public_addr":"172.21.15.145:6825/82446","cluster_addr":"172.21.15.145:6827/82446","heartbeat_back_addr":"172.21.15.145:6831/82446","heartbeat_front_addr":"172.21.15.145:6829/82446","state":["exists","up"]},{"osd":2,"uuid":"e10a1379-65a2-40f5-9308-9385c268cbbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":82445},{"type":"v1","addr":"172.21.15.145:6809","nonce":82445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":82445},{"type":"v1","addr":"172.21.15.145:6811","nonce":82445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":82445},{"type":"v1","addr":"172.21.15.145:6815","nonce":82445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":82445},{"type":"v1","addr":"172.21.15.145:6813","nonce":82445}]},"public_addr":"172.21.15.145:6809/82445","cluster_addr":"172.21.15.145:6811/82445","heartbeat_back_addr":"172.21.15.145:6815/82445","heartbeat_front_addr":"172.21.15.145:6813/82445","state":["exists","up"]},{"osd":3,"uuid":"294a4836-c07e-4229-9d35-ef1211b721a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":82456},{"type":"v1","addr":"172.21.15.145:6817","nonce":82456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":82456},{"type":"v1","addr":"172.21.15.145:6819","nonce":82456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":82456},{"type":"v1","addr":"172.21.15.145:6823","nonce":82456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":82456},{"type":"v1","addr":"172.21.15.145:6821","nonce":82456}]},"public_addr":"172.21.15.145:6817/82456","cluster_addr":"172.21.15.145:6819/82456","heartbeat_back_addr":"172.21.15.145:6823/82456","heartbeat_front_addr":"172.21.15.145:6821/82456","state":["exists","up"]},{"osd":4,"uuid":"43f5962f-28f4-4970-a0e0-95f9ab3a1d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":82396},{"type":"v1","addr":"172.21.15.167:6809","nonce":82396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":82396},{"type":"v1","addr":"172.21.15.167:6811","nonce":82396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":82396},{"type":"v1","addr":"172.21.15.167:6815","nonce":82396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":82396},{"type":"v1","addr":"172.21.15.167:6813","nonce":82396}]},"public_addr":"172.21.15.167:6809/82396","cluster_addr":"172.21.15.167:6811/82396","heartbeat_back_addr":"172.21.15.167:6815/82396","heartbeat_front_addr":"172.21.15.167:6813/82396","state":["exists","up"]},{"osd":5,"uuid":"dbfc9c39-2a9a-496f-b135-c7456357985d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":82395},{"type":"v1","addr":"172.21.15.167:6801","nonce":82395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":82395},{"type":"v1","addr":"172.21.15.167:6803","nonce":82395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":82395},{"type":"v1","addr":"172.21.15.167:6807","nonce":82395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":82395},{"type":"v1","addr":"172.21.15.167:6805","nonce":82395}]},"public_addr":"172.21.15.167:6801/82395","cluster_addr":"172.21.15.167:6803/82395","heartbeat_back_addr":"172.21.15.167:6807/82395","heartbeat_front_addr":"172.21.15.167:6805/82395","state":["exists","up"]},{"osd":6,"uuid":"8d0a58d6-f8e1-4728-922b-a360c114f939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":82425},{"type":"v1","addr":"172.21.15.167:6817","nonce":82425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":82425},{"type":"v1","addr":"172.21.15.167:6819","nonce":82425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":82425},{"type":"v1","addr":"172.21.15.167:6823","nonce":82425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":82425},{"type":"v1","addr":"172.21.15.167:6821","nonce":82425}]},"public_addr":"172.21.15.167:6817/82425","cluster_addr":"172.21.15.167:6819/82425","heartbeat_back_addr":"172.21.15.167:6823/82425","heartbeat_front_addr":"172.21.15.167:6821/82425","state":["exists","up"]},{"osd":7,"uuid":"3a05c01e-9ac9-4e3f-a173-7dc4997671e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":82426},{"type":"v1","addr":"172.21.15.167:6825","nonce":82426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":82426},{"type":"v1","addr":"172.21.15.167:6827","nonce":82426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":82426},{"type":"v1","addr":"172.21.15.167:6831","nonce":82426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":82426},{"type":"v1","addr":"172.21.15.167:6829","nonce":82426}]},"public_addr":"172.21.15.167:6825/82426","cluster_addr":"172.21.15.167:6827/82426","heartbeat_back_addr":"172.21.15.167:6831/82426","heartbeat_front_addr":"172.21.15.167:6829/82426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.328149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.297740+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.330892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.357625+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.124789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.034711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.331042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.366351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:52:22.540 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T18:52:22.540 INFO:tasks.ceph:Creating RBD pool 2023-01-24T18:52:22.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T18:52:23.610 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'rbd' created 2023-01-24T18:52:23.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T18:52:24.612 INFO:teuthology.orchestra.run.smithi145.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T18:52:24.625 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T18:52:24.625 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T18:52:24.626 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T18:52:24.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T18:52:24.627 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T18:52:24.987 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:24.999 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":81840},{"type":"v1","addr":"172.21.15.167:6833","nonce":81840}]},"active_addr":"172.21.15.167:6833/81840","active_change":"2023-01-24T18:52:17.169722+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4112,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2842253383}]},{"addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2852435209}]},{"addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1160499285}]}]}} 2023-01-24T18:52:25.003 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T18:52:25.003 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T18:52:25.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:52:25.307 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:25.308 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"fsid":"f7057319-7755-43a0-abfd-bc543eb5d39e","created":"2023-01-24T18:52:11.784905+0000","modified":"2023-01-24T18:52:24.602277+0000","last_up_change":"2023-01-24T18:52:20.579539+0000","last_in_change":"2023-01-24T18:52:14.150744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:52:21.206164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T18:52:22.860236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"20f62281-4976-49fd-9fcc-50fdeb27053c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":82439},{"type":"v1","addr":"172.21.15.145:6801","nonce":82439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":82439},{"type":"v1","addr":"172.21.15.145:6803","nonce":82439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":82439},{"type":"v1","addr":"172.21.15.145:6807","nonce":82439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":82439},{"type":"v1","addr":"172.21.15.145:6805","nonce":82439}]},"public_addr":"172.21.15.145:6801/82439","cluster_addr":"172.21.15.145:6803/82439","heartbeat_back_addr":"172.21.15.145:6807/82439","heartbeat_front_addr":"172.21.15.145:6805/82439","state":["exists","up"]},{"osd":1,"uuid":"b78a83ab-291c-40d9-8b6e-d65ca043d34a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":82446},{"type":"v1","addr":"172.21.15.145:6825","nonce":82446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":82446},{"type":"v1","addr":"172.21.15.145:6827","nonce":82446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":82446},{"type":"v1","addr":"172.21.15.145:6831","nonce":82446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":82446},{"type":"v1","addr":"172.21.15.145:6829","nonce":82446}]},"public_addr":"172.21.15.145:6825/82446","cluster_addr":"172.21.15.145:6827/82446","heartbeat_back_addr":"172.21.15.145:6831/82446","heartbeat_front_addr":"172.21.15.145:6829/82446","state":["exists","up"]},{"osd":2,"uuid":"e10a1379-65a2-40f5-9308-9385c268cbbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":82445},{"type":"v1","addr":"172.21.15.145:6809","nonce":82445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":82445},{"type":"v1","addr":"172.21.15.145:6811","nonce":82445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":82445},{"type":"v1","addr":"172.21.15.145:6815","nonce":82445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":82445},{"type":"v1","addr":"172.21.15.145:6813","nonce":82445}]},"public_addr":"172.21.15.145:6809/82445","cluster_addr":"172.21.15.145:6811/82445","heartbeat_back_addr":"172.21.15.145:6815/82445","heartbeat_front_addr":"172.21.15.145:6813/82445","state":["exists","up"]},{"osd":3,"uuid":"294a4836-c07e-4229-9d35-ef1211b721a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":82456},{"type":"v1","addr":"172.21.15.145:6817","nonce":82456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":82456},{"type":"v1","addr":"172.21.15.145:6819","nonce":82456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":82456},{"type":"v1","addr":"172.21.15.145:6823","nonce":82456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":82456},{"type":"v1","addr":"172.21.15.145:6821","nonce":82456}]},"public_addr":"172.21.15.145:6817/82456","cluster_addr":"172.21.15.145:6819/82456","heartbeat_back_addr":"172.21.15.145:6823/82456","heartbeat_front_addr":"172.21.15.145:6821/82456","state":["exists","up"]},{"osd":4,"uuid":"43f5962f-28f4-4970-a0e0-95f9ab3a1d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":82396},{"type":"v1","addr":"172.21.15.167:6809","nonce":82396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":82396},{"type":"v1","addr":"172.21.15.167:6811","nonce":82396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":82396},{"type":"v1","addr":"172.21.15.167:6815","nonce":82396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":82396},{"type":"v1","addr":"172.21.15.167:6813","nonce":82396}]},"public_addr":"172.21.15.167:6809/82396","cluster_addr":"172.21.15.167:6811/82396","heartbeat_back_addr":"172.21.15.167:6815/82396","heartbeat_front_addr":"172.21.15.167:6813/82396","state":["exists","up"]},{"osd":5,"uuid":"dbfc9c39-2a9a-496f-b135-c7456357985d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":82395},{"type":"v1","addr":"172.21.15.167:6801","nonce":82395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":82395},{"type":"v1","addr":"172.21.15.167:6803","nonce":82395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":82395},{"type":"v1","addr":"172.21.15.167:6807","nonce":82395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":82395},{"type":"v1","addr":"172.21.15.167:6805","nonce":82395}]},"public_addr":"172.21.15.167:6801/82395","cluster_addr":"172.21.15.167:6803/82395","heartbeat_back_addr":"172.21.15.167:6807/82395","heartbeat_front_addr":"172.21.15.167:6805/82395","state":["exists","up"]},{"osd":6,"uuid":"8d0a58d6-f8e1-4728-922b-a360c114f939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":82425},{"type":"v1","addr":"172.21.15.167:6817","nonce":82425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":82425},{"type":"v1","addr":"172.21.15.167:6819","nonce":82425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":82425},{"type":"v1","addr":"172.21.15.167:6823","nonce":82425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":82425},{"type":"v1","addr":"172.21.15.167:6821","nonce":82425}]},"public_addr":"172.21.15.167:6817/82425","cluster_addr":"172.21.15.167:6819/82425","heartbeat_back_addr":"172.21.15.167:6823/82425","heartbeat_front_addr":"172.21.15.167:6821/82425","state":["exists","up"]},{"osd":7,"uuid":"3a05c01e-9ac9-4e3f-a173-7dc4997671e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":82426},{"type":"v1","addr":"172.21.15.167:6825","nonce":82426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":82426},{"type":"v1","addr":"172.21.15.167:6827","nonce":82426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":82426},{"type":"v1","addr":"172.21.15.167:6831","nonce":82426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":82426},{"type":"v1","addr":"172.21.15.167:6829","nonce":82426}]},"public_addr":"172.21.15.167:6825/82426","cluster_addr":"172.21.15.167:6827/82426","heartbeat_back_addr":"172.21.15.167:6831/82426","heartbeat_front_addr":"172.21.15.167:6829/82426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.328149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.297740+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.330892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.357625+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.124789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.034711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.331042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.366351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:52:25.321 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T18:52:25.322 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:52:25.624 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:25.625 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"fsid":"f7057319-7755-43a0-abfd-bc543eb5d39e","created":"2023-01-24T18:52:11.784905+0000","modified":"2023-01-24T18:52:25.607766+0000","last_up_change":"2023-01-24T18:52:20.579539+0000","last_in_change":"2023-01-24T18:52:14.150744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:52:21.206164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T18:52:22.860236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"20f62281-4976-49fd-9fcc-50fdeb27053c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":82439},{"type":"v1","addr":"172.21.15.145:6801","nonce":82439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":82439},{"type":"v1","addr":"172.21.15.145:6803","nonce":82439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":82439},{"type":"v1","addr":"172.21.15.145:6807","nonce":82439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":82439},{"type":"v1","addr":"172.21.15.145:6805","nonce":82439}]},"public_addr":"172.21.15.145:6801/82439","cluster_addr":"172.21.15.145:6803/82439","heartbeat_back_addr":"172.21.15.145:6807/82439","heartbeat_front_addr":"172.21.15.145:6805/82439","state":["exists","up"]},{"osd":1,"uuid":"b78a83ab-291c-40d9-8b6e-d65ca043d34a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":82446},{"type":"v1","addr":"172.21.15.145:6825","nonce":82446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":82446},{"type":"v1","addr":"172.21.15.145:6827","nonce":82446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":82446},{"type":"v1","addr":"172.21.15.145:6831","nonce":82446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":82446},{"type":"v1","addr":"172.21.15.145:6829","nonce":82446}]},"public_addr":"172.21.15.145:6825/82446","cluster_addr":"172.21.15.145:6827/82446","heartbeat_back_addr":"172.21.15.145:6831/82446","heartbeat_front_addr":"172.21.15.145:6829/82446","state":["exists","up"]},{"osd":2,"uuid":"e10a1379-65a2-40f5-9308-9385c268cbbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":82445},{"type":"v1","addr":"172.21.15.145:6809","nonce":82445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":82445},{"type":"v1","addr":"172.21.15.145:6811","nonce":82445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":82445},{"type":"v1","addr":"172.21.15.145:6815","nonce":82445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":82445},{"type":"v1","addr":"172.21.15.145:6813","nonce":82445}]},"public_addr":"172.21.15.145:6809/82445","cluster_addr":"172.21.15.145:6811/82445","heartbeat_back_addr":"172.21.15.145:6815/82445","heartbeat_front_addr":"172.21.15.145:6813/82445","state":["exists","up"]},{"osd":3,"uuid":"294a4836-c07e-4229-9d35-ef1211b721a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":82456},{"type":"v1","addr":"172.21.15.145:6817","nonce":82456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":82456},{"type":"v1","addr":"172.21.15.145:6819","nonce":82456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":82456},{"type":"v1","addr":"172.21.15.145:6823","nonce":82456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":82456},{"type":"v1","addr":"172.21.15.145:6821","nonce":82456}]},"public_addr":"172.21.15.145:6817/82456","cluster_addr":"172.21.15.145:6819/82456","heartbeat_back_addr":"172.21.15.145:6823/82456","heartbeat_front_addr":"172.21.15.145:6821/82456","state":["exists","up"]},{"osd":4,"uuid":"43f5962f-28f4-4970-a0e0-95f9ab3a1d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":82396},{"type":"v1","addr":"172.21.15.167:6809","nonce":82396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":82396},{"type":"v1","addr":"172.21.15.167:6811","nonce":82396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":82396},{"type":"v1","addr":"172.21.15.167:6815","nonce":82396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":82396},{"type":"v1","addr":"172.21.15.167:6813","nonce":82396}]},"public_addr":"172.21.15.167:6809/82396","cluster_addr":"172.21.15.167:6811/82396","heartbeat_back_addr":"172.21.15.167:6815/82396","heartbeat_front_addr":"172.21.15.167:6813/82396","state":["exists","up"]},{"osd":5,"uuid":"dbfc9c39-2a9a-496f-b135-c7456357985d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":82395},{"type":"v1","addr":"172.21.15.167:6801","nonce":82395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":82395},{"type":"v1","addr":"172.21.15.167:6803","nonce":82395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":82395},{"type":"v1","addr":"172.21.15.167:6807","nonce":82395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":82395},{"type":"v1","addr":"172.21.15.167:6805","nonce":82395}]},"public_addr":"172.21.15.167:6801/82395","cluster_addr":"172.21.15.167:6803/82395","heartbeat_back_addr":"172.21.15.167:6807/82395","heartbeat_front_addr":"172.21.15.167:6805/82395","state":["exists","up"]},{"osd":6,"uuid":"8d0a58d6-f8e1-4728-922b-a360c114f939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":82425},{"type":"v1","addr":"172.21.15.167:6817","nonce":82425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":82425},{"type":"v1","addr":"172.21.15.167:6819","nonce":82425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":82425},{"type":"v1","addr":"172.21.15.167:6823","nonce":82425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":82425},{"type":"v1","addr":"172.21.15.167:6821","nonce":82425}]},"public_addr":"172.21.15.167:6817/82425","cluster_addr":"172.21.15.167:6819/82425","heartbeat_back_addr":"172.21.15.167:6823/82425","heartbeat_front_addr":"172.21.15.167:6821/82425","state":["exists","up"]},{"osd":7,"uuid":"3a05c01e-9ac9-4e3f-a173-7dc4997671e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":82426},{"type":"v1","addr":"172.21.15.167:6825","nonce":82426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":82426},{"type":"v1","addr":"172.21.15.167:6827","nonce":82426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":82426},{"type":"v1","addr":"172.21.15.167:6831","nonce":82426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":82426},{"type":"v1","addr":"172.21.15.167:6829","nonce":82426}]},"public_addr":"172.21.15.167:6825/82426","cluster_addr":"172.21.15.167:6827/82426","heartbeat_back_addr":"172.21.15.167:6831/82426","heartbeat_front_addr":"172.21.15.167:6829/82426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.328149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.297740+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.330892+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.357625+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.124789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.034711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.331042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:52:19.366351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:52:25.639 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T18:52:25.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T18:52:25.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T18:52:25.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T18:52:25.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T18:52:25.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T18:52:25.642 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T18:52:25.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T18:52:25.904 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.042 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T18:52:26.043 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.044 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T18:52:26.047 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.048 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T18:52:26.051 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:52:26.052 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.053 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T18:52:26.054 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.054 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T18:52:26.055 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:26.055 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T18:52:26.056 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T18:52:26.633 INFO:teuthology.orchestra.run.smithi145.stdout:55834574849 2023-01-24T18:52:26.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.5 2023-01-24T18:52:26.680 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.688 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.688 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.692 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.693 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.698 INFO:teuthology.orchestra.run.smithi145.stdout:55834574850 2023-01-24T18:52:26.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-01-24T18:52:26.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-24T18:52:26.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-24T18:52:26.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-24T18:52:26.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-24T18:52:26.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-24T18:52:26.726 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:26.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-01-24T18:52:26.793 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-01-24T18:52:26.791+0000 7faf676d0700 -1 osd.5 17 handle_osd_map injecting map crc failure 2023-01-24T18:52:27.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T18:52:27.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T18:52:27.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:52:27.715 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T18:52:27.718 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T18:52:27.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T18:52:27.726 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T18:52:27.742 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T18:52:28.226 INFO:teuthology.orchestra.run.smithi145.stdout:55834574851 2023-01-24T18:52:28.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.5 2023-01-24T18:52:28.251 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.294 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.312 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.0 2023-01-24T18:52:28.319 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.319 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2023-01-24T18:52:28.334 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.340 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.353 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2023-01-24T18:52:28.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2023-01-24T18:52:28.355 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2023-01-24T18:52:28.362 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.370 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2023-01-24T18:52:28.376 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.7 2023-01-24T18:52:28.383 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.392 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2023-01-24T18:52:28.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.1 2023-01-24T18:52:28.406 DEBUG:teuthology.parallel:result is None 2023-01-24T18:52:28.407 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:52:28.407 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:52:28.746 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:28.748 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-01-24T18:52:28.760 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T18:52:27.175911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40788,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":754933932,"statfs":{"total":773094113280,"available":773052346368,"internally_reserved":0,"allocated":1314816,"data_stored":694584,"data_compressed":12730,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.612761+0000","last_change":"2023-01-24T18:52:24.612761+0000","last_active":"2023-01-24T18:52:24.612761+0000","last_peered":"2023-01-24T18:52:24.612761+0000","last_clean":"2023-01-24T18:52:24.612761+0000","last_became_active":"2023-01-24T18:52:24.612451+0000","last_became_peered":"2023-01-24T18:52:24.612451+0000","last_unstale":"2023-01-24T18:52:24.612761+0000","last_undegraded":"2023-01-24T18:52:24.612761+0000","last_fullsized":"2023-01-24T18:52:24.612761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.910276+0000","last_change":"2023-01-24T18:52:24.809774+0000","last_active":"2023-01-24T18:52:24.910276+0000","last_peered":"2023-01-24T18:52:24.910276+0000","last_clean":"2023-01-24T18:52:24.910276+0000","last_became_active":"2023-01-24T18:52:24.809556+0000","last_became_peered":"2023-01-24T18:52:24.809556+0000","last_unstale":"2023-01-24T18:52:24.910276+0000","last_undegraded":"2023-01-24T18:52:24.910276+0000","last_fullsized":"2023-01-24T18:52:24.910276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:43.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.679005+0000","last_change":"2023-01-24T18:52:24.679005+0000","last_active":"2023-01-24T18:52:24.679005+0000","last_peered":"2023-01-24T18:52:24.679005+0000","last_clean":"2023-01-24T18:52:24.679005+0000","last_became_active":"2023-01-24T18:52:24.678686+0000","last_became_peered":"2023-01-24T18:52:24.678686+0000","last_unstale":"2023-01-24T18:52:24.679005+0000","last_undegraded":"2023-01-24T18:52:24.679005+0000","last_fullsized":"2023-01-24T18:52:24.679005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:52.530716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.712509+0000","last_change":"2023-01-24T18:52:24.712509+0000","last_active":"2023-01-24T18:52:24.712509+0000","last_peered":"2023-01-24T18:52:24.712509+0000","last_clean":"2023-01-24T18:52:24.712509+0000","last_became_active":"2023-01-24T18:52:24.712236+0000","last_became_peered":"2023-01-24T18:52:24.712236+0000","last_unstale":"2023-01-24T18:52:24.712509+0000","last_undegraded":"2023-01-24T18:52:24.712509+0000","last_fullsized":"2023-01-24T18:52:24.712509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.855068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'37","reported_seq":62,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:26.541956+0000","last_change":"2023-01-24T18:52:23.153172+0000","last_active":"2023-01-24T18:52:26.541956+0000","last_peered":"2023-01-24T18:52:26.541956+0000","last_clean":"2023-01-24T18:52:26.541956+0000","last_became_active":"2023-01-24T18:52:23.152349+0000","last_became_peered":"2023-01-24T18:52:23.152349+0000","last_unstale":"2023-01-24T18:52:26.541956+0000","last_undegraded":"2023-01-24T18:52:26.541956+0000","last_fullsized":"2023-01-24T18:52:26.541956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_clean_scrub_stamp":"2023-01-24T18:52:21.583855+0000","objects_scrubbed":0,"log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:37.960457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.708945+0000","last_change":"2023-01-24T18:52:24.708945+0000","last_active":"2023-01-24T18:52:24.708945+0000","last_peered":"2023-01-24T18:52:24.708945+0000","last_clean":"2023-01-24T18:52:24.708945+0000","last_became_active":"2023-01-24T18:52:24.708704+0000","last_became_peered":"2023-01-24T18:52:24.708704+0000","last_unstale":"2023-01-24T18:52:24.708945+0000","last_undegraded":"2023-01-24T18:52:24.708945+0000","last_fullsized":"2023-01-24T18:52:24.708945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:41.340494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.709940+0000","last_change":"2023-01-24T18:52:24.609096+0000","last_active":"2023-01-24T18:52:24.709940+0000","last_peered":"2023-01-24T18:52:24.709940+0000","last_clean":"2023-01-24T18:52:24.709940+0000","last_became_active":"2023-01-24T18:52:24.608813+0000","last_became_peered":"2023-01-24T18:52:24.608813+0000","last_unstale":"2023-01-24T18:52:24.709940+0000","last_undegraded":"2023-01-24T18:52:24.709940+0000","last_fullsized":"2023-01-24T18:52:24.709940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.611091+0000","last_change":"2023-01-24T18:52:24.611091+0000","last_active":"2023-01-24T18:52:24.611091+0000","last_peered":"2023-01-24T18:52:24.611091+0000","last_clean":"2023-01-24T18:52:24.611091+0000","last_became_active":"2023-01-24T18:52:24.610818+0000","last_became_peered":"2023-01-24T18:52:24.610818+0000","last_unstale":"2023-01-24T18:52:24.611091+0000","last_undegraded":"2023-01-24T18:52:24.611091+0000","last_fullsized":"2023-01-24T18:52:24.611091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:25.139189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.781902+0000","last_change":"2023-01-24T18:52:24.778846+0000","last_active":"2023-01-24T18:52:24.781902+0000","last_peered":"2023-01-24T18:52:24.781902+0000","last_clean":"2023-01-24T18:52:24.781902+0000","last_became_active":"2023-01-24T18:52:24.778521+0000","last_became_peered":"2023-01-24T18:52:24.778521+0000","last_unstale":"2023-01-24T18:52:24.781902+0000","last_undegraded":"2023-01-24T18:52:24.781902+0000","last_fullsized":"2023-01-24T18:52:24.781902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.489445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":410656,"data_compressed":19026,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5192,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366648,"statfs":{"total":96636764160,"available":96631447552,"internally_reserved":0,"allocated":188416,"data_stored":191287,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3159999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":189015,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":48379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:52:28.761 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:52:29.058 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:29.061 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-01-24T18:52:29.071 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T18:52:27.175911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40788,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":754933932,"statfs":{"total":773094113280,"available":773052346368,"internally_reserved":0,"allocated":1314816,"data_stored":694584,"data_compressed":12730,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.612761+0000","last_change":"2023-01-24T18:52:24.612761+0000","last_active":"2023-01-24T18:52:24.612761+0000","last_peered":"2023-01-24T18:52:24.612761+0000","last_clean":"2023-01-24T18:52:24.612761+0000","last_became_active":"2023-01-24T18:52:24.612451+0000","last_became_peered":"2023-01-24T18:52:24.612451+0000","last_unstale":"2023-01-24T18:52:24.612761+0000","last_undegraded":"2023-01-24T18:52:24.612761+0000","last_fullsized":"2023-01-24T18:52:24.612761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.910276+0000","last_change":"2023-01-24T18:52:24.809774+0000","last_active":"2023-01-24T18:52:24.910276+0000","last_peered":"2023-01-24T18:52:24.910276+0000","last_clean":"2023-01-24T18:52:24.910276+0000","last_became_active":"2023-01-24T18:52:24.809556+0000","last_became_peered":"2023-01-24T18:52:24.809556+0000","last_unstale":"2023-01-24T18:52:24.910276+0000","last_undegraded":"2023-01-24T18:52:24.910276+0000","last_fullsized":"2023-01-24T18:52:24.910276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:43.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.679005+0000","last_change":"2023-01-24T18:52:24.679005+0000","last_active":"2023-01-24T18:52:24.679005+0000","last_peered":"2023-01-24T18:52:24.679005+0000","last_clean":"2023-01-24T18:52:24.679005+0000","last_became_active":"2023-01-24T18:52:24.678686+0000","last_became_peered":"2023-01-24T18:52:24.678686+0000","last_unstale":"2023-01-24T18:52:24.679005+0000","last_undegraded":"2023-01-24T18:52:24.679005+0000","last_fullsized":"2023-01-24T18:52:24.679005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:52.530716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.712509+0000","last_change":"2023-01-24T18:52:24.712509+0000","last_active":"2023-01-24T18:52:24.712509+0000","last_peered":"2023-01-24T18:52:24.712509+0000","last_clean":"2023-01-24T18:52:24.712509+0000","last_became_active":"2023-01-24T18:52:24.712236+0000","last_became_peered":"2023-01-24T18:52:24.712236+0000","last_unstale":"2023-01-24T18:52:24.712509+0000","last_undegraded":"2023-01-24T18:52:24.712509+0000","last_fullsized":"2023-01-24T18:52:24.712509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.855068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'37","reported_seq":62,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:26.541956+0000","last_change":"2023-01-24T18:52:23.153172+0000","last_active":"2023-01-24T18:52:26.541956+0000","last_peered":"2023-01-24T18:52:26.541956+0000","last_clean":"2023-01-24T18:52:26.541956+0000","last_became_active":"2023-01-24T18:52:23.152349+0000","last_became_peered":"2023-01-24T18:52:23.152349+0000","last_unstale":"2023-01-24T18:52:26.541956+0000","last_undegraded":"2023-01-24T18:52:26.541956+0000","last_fullsized":"2023-01-24T18:52:26.541956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_clean_scrub_stamp":"2023-01-24T18:52:21.583855+0000","objects_scrubbed":0,"log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:37.960457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.708945+0000","last_change":"2023-01-24T18:52:24.708945+0000","last_active":"2023-01-24T18:52:24.708945+0000","last_peered":"2023-01-24T18:52:24.708945+0000","last_clean":"2023-01-24T18:52:24.708945+0000","last_became_active":"2023-01-24T18:52:24.708704+0000","last_became_peered":"2023-01-24T18:52:24.708704+0000","last_unstale":"2023-01-24T18:52:24.708945+0000","last_undegraded":"2023-01-24T18:52:24.708945+0000","last_fullsized":"2023-01-24T18:52:24.708945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:41.340494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.709940+0000","last_change":"2023-01-24T18:52:24.609096+0000","last_active":"2023-01-24T18:52:24.709940+0000","last_peered":"2023-01-24T18:52:24.709940+0000","last_clean":"2023-01-24T18:52:24.709940+0000","last_became_active":"2023-01-24T18:52:24.608813+0000","last_became_peered":"2023-01-24T18:52:24.608813+0000","last_unstale":"2023-01-24T18:52:24.709940+0000","last_undegraded":"2023-01-24T18:52:24.709940+0000","last_fullsized":"2023-01-24T18:52:24.709940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.611091+0000","last_change":"2023-01-24T18:52:24.611091+0000","last_active":"2023-01-24T18:52:24.611091+0000","last_peered":"2023-01-24T18:52:24.611091+0000","last_clean":"2023-01-24T18:52:24.611091+0000","last_became_active":"2023-01-24T18:52:24.610818+0000","last_became_peered":"2023-01-24T18:52:24.610818+0000","last_unstale":"2023-01-24T18:52:24.611091+0000","last_undegraded":"2023-01-24T18:52:24.611091+0000","last_fullsized":"2023-01-24T18:52:24.611091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:25.139189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.781902+0000","last_change":"2023-01-24T18:52:24.778846+0000","last_active":"2023-01-24T18:52:24.781902+0000","last_peered":"2023-01-24T18:52:24.781902+0000","last_clean":"2023-01-24T18:52:24.781902+0000","last_became_active":"2023-01-24T18:52:24.778521+0000","last_became_peered":"2023-01-24T18:52:24.778521+0000","last_unstale":"2023-01-24T18:52:24.781902+0000","last_undegraded":"2023-01-24T18:52:24.781902+0000","last_fullsized":"2023-01-24T18:52:24.781902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.489445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":410656,"data_compressed":19026,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5192,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366648,"statfs":{"total":96636764160,"available":96631447552,"internally_reserved":0,"allocated":188416,"data_stored":191287,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3159999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":189015,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":48379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:52:29.073 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T18:52:29.073 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T18:52:29.073 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T18:52:29.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T18:52:29.447 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:29.447 INFO:teuthology.orchestra.run.smithi145.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T18:52:29.461 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T18:52:29.461 INFO:teuthology.run_tasks:Running task exec... 2023-01-24T18:52:29.471 INFO:teuthology.task.exec:Executing custom commands... 2023-01-24T18:52:29.472 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi145.front.sepia.ceph.com 2023-01-24T18:52:29.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2023-01-24T18:52:29.789 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-01-24T18:52:29.790 INFO:teuthology.orchestra.run.smithi145.stdout: "active": true, 2023-01-24T18:52:29.790 INFO:teuthology.orchestra.run.smithi145.stdout: "last_optimize_duration": "0:00:00.000372", 2023-01-24T18:52:29.790 INFO:teuthology.orchestra.run.smithi145.stdout: "last_optimize_started": "Tue Jan 24 18:52:17 2023", 2023-01-24T18:52:29.790 INFO:teuthology.orchestra.run.smithi145.stdout: "mode": "upmap", 2023-01-24T18:52:29.791 INFO:teuthology.orchestra.run.smithi145.stdout: "no_optimization_needed": false, 2023-01-24T18:52:29.791 INFO:teuthology.orchestra.run.smithi145.stdout: "optimize_result": "No pools available", 2023-01-24T18:52:29.791 INFO:teuthology.orchestra.run.smithi145.stdout: "plans": [] 2023-01-24T18:52:29.791 INFO:teuthology.orchestra.run.smithi145.stdout:} 2023-01-24T18:52:29.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2023-01-24T18:52:30.690 DEBUG:teuthology.orchestra.run.smithi145:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2023-01-24T18:52:30.815 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-01-24T18:52:30.814+0000 7f7f24c78700 -1 osd.7 18 handle_osd_map injecting map crc failure 2023-01-24T18:52:31.003 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-24T18:52:31.023 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-24T18:52:31.023 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T18:52:31.023 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2023-01-24T18:52:31.024 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-24T18:52:31.363 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-24T18:52:31.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:52:31.364 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:52:31.670 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:31.670 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-01-24T18:52:31.680 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-01-24T18:52:31.176720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40788,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":754933932,"statfs":{"total":773094113280,"available":773052346368,"internally_reserved":0,"allocated":1314816,"data_stored":694584,"data_compressed":12730,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000808"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.612761+0000","last_change":"2023-01-24T18:52:24.612761+0000","last_active":"2023-01-24T18:52:24.612761+0000","last_peered":"2023-01-24T18:52:24.612761+0000","last_clean":"2023-01-24T18:52:24.612761+0000","last_became_active":"2023-01-24T18:52:24.612451+0000","last_became_peered":"2023-01-24T18:52:24.612451+0000","last_unstale":"2023-01-24T18:52:24.612761+0000","last_undegraded":"2023-01-24T18:52:24.612761+0000","last_fullsized":"2023-01-24T18:52:24.612761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.910276+0000","last_change":"2023-01-24T18:52:24.809774+0000","last_active":"2023-01-24T18:52:24.910276+0000","last_peered":"2023-01-24T18:52:24.910276+0000","last_clean":"2023-01-24T18:52:24.910276+0000","last_became_active":"2023-01-24T18:52:24.809556+0000","last_became_peered":"2023-01-24T18:52:24.809556+0000","last_unstale":"2023-01-24T18:52:24.910276+0000","last_undegraded":"2023-01-24T18:52:24.910276+0000","last_fullsized":"2023-01-24T18:52:24.910276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:43.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.679005+0000","last_change":"2023-01-24T18:52:24.679005+0000","last_active":"2023-01-24T18:52:24.679005+0000","last_peered":"2023-01-24T18:52:24.679005+0000","last_clean":"2023-01-24T18:52:24.679005+0000","last_became_active":"2023-01-24T18:52:24.678686+0000","last_became_peered":"2023-01-24T18:52:24.678686+0000","last_unstale":"2023-01-24T18:52:24.679005+0000","last_undegraded":"2023-01-24T18:52:24.679005+0000","last_fullsized":"2023-01-24T18:52:24.679005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:52.530716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.712509+0000","last_change":"2023-01-24T18:52:24.712509+0000","last_active":"2023-01-24T18:52:24.712509+0000","last_peered":"2023-01-24T18:52:24.712509+0000","last_clean":"2023-01-24T18:52:24.712509+0000","last_became_active":"2023-01-24T18:52:24.712236+0000","last_became_peered":"2023-01-24T18:52:24.712236+0000","last_unstale":"2023-01-24T18:52:24.712509+0000","last_undegraded":"2023-01-24T18:52:24.712509+0000","last_fullsized":"2023-01-24T18:52:24.712509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.855068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'37","reported_seq":62,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:26.541956+0000","last_change":"2023-01-24T18:52:23.153172+0000","last_active":"2023-01-24T18:52:26.541956+0000","last_peered":"2023-01-24T18:52:26.541956+0000","last_clean":"2023-01-24T18:52:26.541956+0000","last_became_active":"2023-01-24T18:52:23.152349+0000","last_became_peered":"2023-01-24T18:52:23.152349+0000","last_unstale":"2023-01-24T18:52:26.541956+0000","last_undegraded":"2023-01-24T18:52:26.541956+0000","last_fullsized":"2023-01-24T18:52:26.541956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_clean_scrub_stamp":"2023-01-24T18:52:21.583855+0000","objects_scrubbed":0,"log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:37.960457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.708945+0000","last_change":"2023-01-24T18:52:24.708945+0000","last_active":"2023-01-24T18:52:24.708945+0000","last_peered":"2023-01-24T18:52:24.708945+0000","last_clean":"2023-01-24T18:52:24.708945+0000","last_became_active":"2023-01-24T18:52:24.708704+0000","last_became_peered":"2023-01-24T18:52:24.708704+0000","last_unstale":"2023-01-24T18:52:24.708945+0000","last_undegraded":"2023-01-24T18:52:24.708945+0000","last_fullsized":"2023-01-24T18:52:24.708945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:41.340494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.709940+0000","last_change":"2023-01-24T18:52:24.609096+0000","last_active":"2023-01-24T18:52:24.709940+0000","last_peered":"2023-01-24T18:52:24.709940+0000","last_clean":"2023-01-24T18:52:24.709940+0000","last_became_active":"2023-01-24T18:52:24.608813+0000","last_became_peered":"2023-01-24T18:52:24.608813+0000","last_unstale":"2023-01-24T18:52:24.709940+0000","last_undegraded":"2023-01-24T18:52:24.709940+0000","last_fullsized":"2023-01-24T18:52:24.709940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.611091+0000","last_change":"2023-01-24T18:52:24.611091+0000","last_active":"2023-01-24T18:52:24.611091+0000","last_peered":"2023-01-24T18:52:24.611091+0000","last_clean":"2023-01-24T18:52:24.611091+0000","last_became_active":"2023-01-24T18:52:24.610818+0000","last_became_peered":"2023-01-24T18:52:24.610818+0000","last_unstale":"2023-01-24T18:52:24.611091+0000","last_undegraded":"2023-01-24T18:52:24.611091+0000","last_fullsized":"2023-01-24T18:52:24.611091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:25.139189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.781902+0000","last_change":"2023-01-24T18:52:24.778846+0000","last_active":"2023-01-24T18:52:24.781902+0000","last_peered":"2023-01-24T18:52:24.781902+0000","last_clean":"2023-01-24T18:52:24.781902+0000","last_became_active":"2023-01-24T18:52:24.778521+0000","last_became_peered":"2023-01-24T18:52:24.778521+0000","last_unstale":"2023-01-24T18:52:24.781902+0000","last_undegraded":"2023-01-24T18:52:24.781902+0000","last_fullsized":"2023-01-24T18:52:24.781902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.489445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":410656,"data_compressed":19026,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5192,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366648,"statfs":{"total":96636764160,"available":96631447552,"internally_reserved":0,"allocated":188416,"data_stored":191287,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3159999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":189015,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":48379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:52:31.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:52:31.982 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-01-24T18:52:31.985 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2023-01-24T18:52:31.996 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-01-24T18:52:31.176720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40788,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":754933932,"statfs":{"total":773094113280,"available":773052346368,"internally_reserved":0,"allocated":1314816,"data_stored":694584,"data_compressed":12730,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000808"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.612761+0000","last_change":"2023-01-24T18:52:24.612761+0000","last_active":"2023-01-24T18:52:24.612761+0000","last_peered":"2023-01-24T18:52:24.612761+0000","last_clean":"2023-01-24T18:52:24.612761+0000","last_became_active":"2023-01-24T18:52:24.612451+0000","last_became_peered":"2023-01-24T18:52:24.612451+0000","last_unstale":"2023-01-24T18:52:24.612761+0000","last_undegraded":"2023-01-24T18:52:24.612761+0000","last_fullsized":"2023-01-24T18:52:24.612761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.910276+0000","last_change":"2023-01-24T18:52:24.809774+0000","last_active":"2023-01-24T18:52:24.910276+0000","last_peered":"2023-01-24T18:52:24.910276+0000","last_clean":"2023-01-24T18:52:24.910276+0000","last_became_active":"2023-01-24T18:52:24.809556+0000","last_became_peered":"2023-01-24T18:52:24.809556+0000","last_unstale":"2023-01-24T18:52:24.910276+0000","last_undegraded":"2023-01-24T18:52:24.910276+0000","last_fullsized":"2023-01-24T18:52:24.910276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:43.991139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.679005+0000","last_change":"2023-01-24T18:52:24.679005+0000","last_active":"2023-01-24T18:52:24.679005+0000","last_peered":"2023-01-24T18:52:24.679005+0000","last_clean":"2023-01-24T18:52:24.679005+0000","last_became_active":"2023-01-24T18:52:24.678686+0000","last_became_peered":"2023-01-24T18:52:24.678686+0000","last_unstale":"2023-01-24T18:52:24.679005+0000","last_undegraded":"2023-01-24T18:52:24.679005+0000","last_fullsized":"2023-01-24T18:52:24.679005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:52.530716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.712509+0000","last_change":"2023-01-24T18:52:24.712509+0000","last_active":"2023-01-24T18:52:24.712509+0000","last_peered":"2023-01-24T18:52:24.712509+0000","last_clean":"2023-01-24T18:52:24.712509+0000","last_became_active":"2023-01-24T18:52:24.712236+0000","last_became_peered":"2023-01-24T18:52:24.712236+0000","last_unstale":"2023-01-24T18:52:24.712509+0000","last_undegraded":"2023-01-24T18:52:24.712509+0000","last_fullsized":"2023-01-24T18:52:24.712509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.855068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'37","reported_seq":62,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:26.541956+0000","last_change":"2023-01-24T18:52:23.153172+0000","last_active":"2023-01-24T18:52:26.541956+0000","last_peered":"2023-01-24T18:52:26.541956+0000","last_clean":"2023-01-24T18:52:26.541956+0000","last_became_active":"2023-01-24T18:52:23.152349+0000","last_became_peered":"2023-01-24T18:52:23.152349+0000","last_unstale":"2023-01-24T18:52:26.541956+0000","last_undegraded":"2023-01-24T18:52:26.541956+0000","last_fullsized":"2023-01-24T18:52:26.541956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:21.583855+0000","last_clean_scrub_stamp":"2023-01-24T18:52:21.583855+0000","objects_scrubbed":0,"log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:37.960457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.708945+0000","last_change":"2023-01-24T18:52:24.708945+0000","last_active":"2023-01-24T18:52:24.708945+0000","last_peered":"2023-01-24T18:52:24.708945+0000","last_clean":"2023-01-24T18:52:24.708945+0000","last_became_active":"2023-01-24T18:52:24.708704+0000","last_became_peered":"2023-01-24T18:52:24.708704+0000","last_unstale":"2023-01-24T18:52:24.708945+0000","last_undegraded":"2023-01-24T18:52:24.708945+0000","last_fullsized":"2023-01-24T18:52:24.708945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:41.340494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.709940+0000","last_change":"2023-01-24T18:52:24.609096+0000","last_active":"2023-01-24T18:52:24.709940+0000","last_peered":"2023-01-24T18:52:24.709940+0000","last_clean":"2023-01-24T18:52:24.709940+0000","last_became_active":"2023-01-24T18:52:24.608813+0000","last_became_peered":"2023-01-24T18:52:24.608813+0000","last_unstale":"2023-01-24T18:52:24.709940+0000","last_undegraded":"2023-01-24T18:52:24.709940+0000","last_fullsized":"2023-01-24T18:52:24.709940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:34.072655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.611091+0000","last_change":"2023-01-24T18:52:24.611091+0000","last_active":"2023-01-24T18:52:24.611091+0000","last_peered":"2023-01-24T18:52:24.611091+0000","last_clean":"2023-01-24T18:52:24.611091+0000","last_became_active":"2023-01-24T18:52:24.610818+0000","last_became_peered":"2023-01-24T18:52:24.610818+0000","last_unstale":"2023-01-24T18:52:24.611091+0000","last_undegraded":"2023-01-24T18:52:24.611091+0000","last_fullsized":"2023-01-24T18:52:24.611091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:25.139189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:52:24.781902+0000","last_change":"2023-01-24T18:52:24.778846+0000","last_active":"2023-01-24T18:52:24.781902+0000","last_peered":"2023-01-24T18:52:24.781902+0000","last_clean":"2023-01-24T18:52:24.781902+0000","last_became_active":"2023-01-24T18:52:24.778521+0000","last_became_peered":"2023-01-24T18:52:24.778521+0000","last_unstale":"2023-01-24T18:52:24.781902+0000","last_undegraded":"2023-01-24T18:52:24.781902+0000","last_fullsized":"2023-01-24T18:52:24.781902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:52:23.597916+0000","last_clean_scrub_stamp":"2023-01-24T18:52:23.597916+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:53:31.489445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":44,"num_write":64,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":410656,"data_compressed":19026,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5192,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366648,"statfs":{"total":96636764160,"available":96631447552,"internally_reserved":0,"allocated":188416,"data_stored":191287,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":49739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9239999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3159999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":189015,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":55475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":48379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":9513,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:52:31.998 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T18:52:31.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-24T18:52:32.301 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 19 2023-01-24T18:52:32.301 INFO:teuthology.orchestra.run.smithi145.stdout:fsid f7057319-7755-43a0-abfd-bc543eb5d39e 2023-01-24T18:52:32.302 INFO:teuthology.orchestra.run.smithi145.stdout:created 2023-01-24T18:52:11.784905+0000 2023-01-24T18:52:32.302 INFO:teuthology.orchestra.run.smithi145.stdout:modified 2023-01-24T18:52:30.659242+0000 2023-01-24T18:52:32.302 INFO:teuthology.orchestra.run.smithi145.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-24T18:52:32.303 INFO:teuthology.orchestra.run.smithi145.stdout:crush_version 5 2023-01-24T18:52:32.303 INFO:teuthology.orchestra.run.smithi145.stdout:full_ratio 0.95 2023-01-24T18:52:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:backfillfull_ratio 0.9 2023-01-24T18:52:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:nearfull_ratio 0.85 2023-01-24T18:52:32.304 INFO:teuthology.orchestra.run.smithi145.stdout:require_min_compat_client luminous 2023-01-24T18:52:32.305 INFO:teuthology.orchestra.run.smithi145.stdout:min_compat_client jewel 2023-01-24T18:52:32.305 INFO:teuthology.orchestra.run.smithi145.stdout:require_osd_release quincy 2023-01-24T18:52:32.306 INFO:teuthology.orchestra.run.smithi145.stdout:stretch_mode_enabled false 2023-01-24T18:52:32.306 INFO:teuthology.orchestra.run.smithi145.stdout:pool 1 '.mgr' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-24T18:52:32.306 INFO:teuthology.orchestra.run.smithi145.stdout:pool 2 'rbd' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 application rbd 2023-01-24T18:52:32.307 INFO:teuthology.orchestra.run.smithi145.stdout:max_osd 8 2023-01-24T18:52:32.307 INFO:teuthology.orchestra.run.smithi145.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/82439,v1:172.21.15.145:6801/82439] [v2:172.21.15.145:6802/82439,v1:172.21.15.145:6803/82439] exists,up 20f62281-4976-49fd-9fcc-50fdeb27053c 2023-01-24T18:52:32.307 INFO:teuthology.orchestra.run.smithi145.stdout:osd.1 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/82446,v1:172.21.15.145:6825/82446] [v2:172.21.15.145:6826/82446,v1:172.21.15.145:6827/82446] exists,up b78a83ab-291c-40d9-8b6e-d65ca043d34a 2023-01-24T18:52:32.307 INFO:teuthology.orchestra.run.smithi145.stdout:osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/82445,v1:172.21.15.145:6809/82445] [v2:172.21.15.145:6810/82445,v1:172.21.15.145:6811/82445] exists,up e10a1379-65a2-40f5-9308-9385c268cbbb 2023-01-24T18:52:32.308 INFO:teuthology.orchestra.run.smithi145.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/82456,v1:172.21.15.145:6817/82456] [v2:172.21.15.145:6818/82456,v1:172.21.15.145:6819/82456] exists,up 294a4836-c07e-4229-9d35-ef1211b721a7 2023-01-24T18:52:32.308 INFO:teuthology.orchestra.run.smithi145.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/82396,v1:172.21.15.167:6809/82396] [v2:172.21.15.167:6810/82396,v1:172.21.15.167:6811/82396] exists,up 43f5962f-28f4-4970-a0e0-95f9ab3a1d47 2023-01-24T18:52:32.308 INFO:teuthology.orchestra.run.smithi145.stdout:osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/82395,v1:172.21.15.167:6801/82395] [v2:172.21.15.167:6802/82395,v1:172.21.15.167:6803/82395] exists,up dbfc9c39-2a9a-496f-b135-c7456357985d 2023-01-24T18:52:32.308 INFO:teuthology.orchestra.run.smithi145.stdout:osd.6 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/82425,v1:172.21.15.167:6817/82425] [v2:172.21.15.167:6818/82425,v1:172.21.15.167:6819/82425] exists,up 8d0a58d6-f8e1-4728-922b-a360c114f939 2023-01-24T18:52:32.309 INFO:teuthology.orchestra.run.smithi145.stdout:osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/82426,v1:172.21.15.167:6825/82426] [v2:172.21.15.167:6826/82426,v1:172.21.15.167:6827/82426] exists,up 3a05c01e-9ac9-4e3f-a173-7dc4997671e0 2023-01-24T18:52:32.315 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/82439,v1:172.21.15.145:6801/82439] [v2:172.21.15.145:6802/82439,v1:172.21.15.145:6803/82439] exists,up 20f62281-4976-49fd-9fcc-50fdeb27053c', 'osd.1 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/82446,v1:172.21.15.145:6825/82446] [v2:172.21.15.145:6826/82446,v1:172.21.15.145:6827/82446] exists,up b78a83ab-291c-40d9-8b6e-d65ca043d34a', 'osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/82445,v1:172.21.15.145:6809/82445] [v2:172.21.15.145:6810/82445,v1:172.21.15.145:6811/82445] exists,up e10a1379-65a2-40f5-9308-9385c268cbbb', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/82456,v1:172.21.15.145:6817/82456] [v2:172.21.15.145:6818/82456,v1:172.21.15.145:6819/82456] exists,up 294a4836-c07e-4229-9d35-ef1211b721a7', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/82396,v1:172.21.15.167:6809/82396] [v2:172.21.15.167:6810/82396,v1:172.21.15.167:6811/82396] exists,up 43f5962f-28f4-4970-a0e0-95f9ab3a1d47', 'osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/82395,v1:172.21.15.167:6801/82395] [v2:172.21.15.167:6802/82395,v1:172.21.15.167:6803/82395] exists,up dbfc9c39-2a9a-496f-b135-c7456357985d', 'osd.6 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/82425,v1:172.21.15.167:6817/82425] [v2:172.21.15.167:6818/82425,v1:172.21.15.167:6819/82425] exists,up 8d0a58d6-f8e1-4728-922b-a360c114f939', 'osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/82426,v1:172.21.15.167:6825/82426] [v2:172.21.15.167:6826/82426,v1:172.21.15.167:6827/82426] exists,up 3a05c01e-9ac9-4e3f-a173-7dc4997671e0'] 2023-01-24T18:52:32.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-24T18:52:32.430 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2023-01-24T18:52:32.430 INFO:teuthology.orchestra.run.smithi145.stdout: "name": "mon.a", 2023-01-24T18:52:32.430 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster": "ceph", 2023-01-24T18:52:32.430 INFO:teuthology.orchestra.run.smithi145.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-24T18:52:32.431 INFO:teuthology.orchestra.run.smithi145.stdout: "admin_socket_mode": "", 2023-01-24T18:52:32.431 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-24T18:52:32.431 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_client_required": "cephx, none", 2023-01-24T18:52:32.431 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_cluster_required": "cephx", 2023-01-24T18:52:32.431 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_debug": "true", 2023-01-24T18:52:32.432 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-24T18:52:32.432 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-24T18:52:32.432 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_service_required": "cephx", 2023-01-24T18:52:32.432 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-24T18:52:32.432 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_supported": "cephx", 2023-01-24T18:52:32.433 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_aio": "true", 2023-01-24T18:52:32.433 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-24T18:52:32.433 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_aio_poll_ms": "250", 2023-01-24T18:52:32.433 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_aio_reap_max": "16", 2023-01-24T18:52:32.433 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_async_discard": "false", 2023-01-24T18:52:32.434 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_block_size": "4096", 2023-01-24T18:52:32.434 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_debug_aio": "false", 2023-01-24T18:52:32.434 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-24T18:52:32.434 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-24T18:52:32.434 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_debug_inflight_ios": "false", 2023-01-24T18:52:32.435 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_enable_discard": "false", 2023-01-24T18:52:32.435 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_flock_retry": "3", 2023-01-24T18:52:32.435 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-24T18:52:32.435 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_inject_crash": "0", 2023-01-24T18:52:32.435 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-24T18:52:32.436 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_ioring": "false", 2023-01-24T18:52:32.436 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_ioring_hipri": "false", 2023-01-24T18:52:32.436 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-24T18:52:32.436 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-24T18:52:32.436 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-24T18:52:32.437 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-24T18:52:32.437 INFO:teuthology.orchestra.run.smithi145.stdout: "bdev_type": "", 2023-01-24T18:52:32.437 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_alloc_size": "1048576", 2023-01-24T18:52:32.437 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_allocator": "hybrid", 2023-01-24T18:52:32.437 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_buffered_io": "true", 2023-01-24T18:52:32.438 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_check_for_zeros": "false", 2023-01-24T18:52:32.438 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-24T18:52:32.438 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-24T18:52:32.438 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_compact_log_sync": "false", 2023-01-24T18:52:32.438 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-24T18:52:32.439 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-24T18:52:32.439 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-24T18:52:32.439 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_max_log_runway": "4194304", 2023-01-24T18:52:32.439 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_max_prefetch": "1048576", 2023-01-24T18:52:32.439 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_min_flush_size": "524288", 2023-01-24T18:52:32.440 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_min_log_runway": "1048576", 2023-01-24T18:52:32.440 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_replay_recovery": "false", 2023-01-24T18:52:32.440 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-24T18:52:32.440 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-24T18:52:32.440 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_sync_write": "false", 2023-01-24T18:52:32.441 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-24T18:52:32.441 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-24T18:52:32.441 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-24T18:52:32.441 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_allocation_from_file": "true", 2023-01-24T18:52:32.441 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_allocator": "hybrid", 2023-01-24T18:52:32.442 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-24T18:52:32.442 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-24T18:52:32.442 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-24T18:52:32.442 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-24T18:52:32.442 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-24T18:52:32.443 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-24T18:52:32.443 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-24T18:52:32.443 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_create": "true", 2023-01-24T18:52:32.443 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_db_create": "false", 2023-01-24T18:52:32.443 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_db_path": "", 2023-01-24T18:52:32.444 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_db_size": "0", 2023-01-24T18:52:32.444 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_path": "", 2023-01-24T18:52:32.444 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_preallocate_file": "false", 2023-01-24T18:52:32.444 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_size": "107374182400", 2023-01-24T18:52:32.444 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_wal_create": "false", 2023-01-24T18:52:32.445 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_wal_path": "", 2023-01-24T18:52:32.445 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_block_wal_size": "100663296", 2023-01-24T18:52:32.445 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bluefs": "true", 2023-01-24T18:52:32.445 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-24T18:52:32.445 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-24T18:52:32.446 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-24T18:52:32.446 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-24T18:52:32.446 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:52:32.446 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:52:32.446 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-24T18:52:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:52:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_autotune": "true", 2023-01-24T18:52:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-24T18:52:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-24T18:52:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-24T18:52:32.448 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-24T18:52:32.448 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_size": "0", 2023-01-24T18:52:32.448 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-24T18:52:32.448 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-24T18:52:32.448 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-24T18:52:32.449 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-24T18:52:32.449 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_type": "2q", 2023-01-24T18:52:32.449 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-24T18:52:32.449 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_clone_cow": "true", 2023-01-24T18:52:32.449 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-24T18:52:32.450 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-24T18:52:32.450 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-24T18:52:32.450 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-24T18:52:32.450 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-24T18:52:32.450 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-24T18:52:32.451 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-24T18:52:32.451 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_mode": "none", 2023-01-24T18:52:32.451 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-24T18:52:32.451 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_csum_type": "crc32c", 2023-01-24T18:52:32.451 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-24T18:52:32.452 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_freelist": "false", 2023-01-24T18:52:32.452 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-24T18:52:32.452 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-24T18:52:32.452 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-24T18:52:32.452 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-24T18:52:32.453 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-24T18:52:32.453 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-24T18:52:32.453 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-24T18:52:32.453 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-24T18:52:32.454 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-24T18:52:32.454 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-24T18:52:32.454 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-24T18:52:32.454 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-24T18:52:32.454 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-24T18:52:32.455 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_small_allocations": "0", 2023-01-24T18:52:32.455 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-24T18:52:32.455 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_default_buffered_read": "true", 2023-01-24T18:52:32.455 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_default_buffered_write": "false", 2023-01-24T18:52:32.455 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-24T18:52:32.456 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-24T18:52:32.456 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-24T18:52:32.456 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-24T18:52:32.456 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-24T18:52:32.456 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-24T18:52:32.457 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-24T18:52:32.457 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-24T18:52:32.457 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fail_eio": "false", 2023-01-24T18:52:32.457 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-24T18:52:32.458 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-24T18:52:32.458 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-24T18:52:32.458 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-24T18:52:32.458 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-24T18:52:32.458 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-24T18:52:32.459 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_mount": "false", 2023-01-24T18:52:32.459 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-24T18:52:32.459 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_umount": "false", 2023-01-24T18:52:32.459 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-24T18:52:32.459 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-24T18:52:32.460 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-24T18:52:32.460 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-24T18:52:32.460 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-24T18:52:32.460 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-24T18:52:32.460 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-24T18:52:32.461 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-24T18:52:32.461 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_ignore_data_csum": "false", 2023-01-24T18:52:32.461 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-24T18:52:32.461 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-24T18:52:32.461 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-24T18:52:32.462 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-24T18:52:32.462 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_log_op_age": "5.000000", 2023-01-24T18:52:32.462 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_alloc_size": "0", 2023-01-24T18:52:32.462 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_blob_size": "0", 2023-01-24T18:52:32.462 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-24T18:52:32.463 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-24T18:52:32.463 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-24T18:52:32.463 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_max_deferred_txc": "32", 2023-01-24T18:52:32.463 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_min_alloc_size": "0", 2023-01-24T18:52:32.463 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-24T18:52:32.464 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-24T18:52:32.464 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_nid_prealloc": "1024", 2023-01-24T18:52:32.464 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-24T18:52:32.464 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-24T18:52:32.465 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-24T18:52:32.465 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-24T18:52:32.465 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_retry_disk_reads": "3", 2023-01-24T18:52:32.465 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_rocksdb_cf": "true", 2023-01-24T18:52:32.465 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-24T18:52:32.466 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2023-01-24T18:52:32.466 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-24T18:52:32.466 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-24T18:52:32.466 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-24T18:52:32.466 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-24T18:52:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_spdk_mem": "512", 2023-01-24T18:52:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-24T18:52:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-24T18:52:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-24T18:52:32.467 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-24T18:52:32.468 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-24T18:52:32.468 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-24T18:52:32.468 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-24T18:52:32.468 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_tracing": "false", 2023-01-24T18:52:32.468 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-24T18:52:32.469 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-24T18:52:32.469 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-24T18:52:32.469 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-24T18:52:32.469 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-24T18:52:32.469 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-24T18:52:32.470 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-24T18:52:32.470 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-24T18:52:32.470 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-24T18:52:32.470 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_zero_block_detection": "false", 2023-01-24T18:52:32.470 INFO:teuthology.orchestra.run.smithi145.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-24T18:52:32.471 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-24T18:52:32.471 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-24T18:52:32.471 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-24T18:52:32.471 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-24T18:52:32.471 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-24T18:52:32.472 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-24T18:52:32.472 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-24T18:52:32.472 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-24T18:52:32.472 INFO:teuthology.orchestra.run.smithi145.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-24T18:52:32.473 INFO:teuthology.orchestra.run.smithi145.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-24T18:52:32.473 INFO:teuthology.orchestra.run.smithi145.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-24T18:52:32.473 INFO:teuthology.orchestra.run.smithi145.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-24T18:52:32.473 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_cluster_require_signatures": "false", 2023-01-24T18:52:32.474 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_cluster_require_version": "2", 2023-01-24T18:52:32.474 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_require_signatures": "false", 2023-01-24T18:52:32.474 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_require_version": "2", 2023-01-24T18:52:32.474 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_service_require_signatures": "false", 2023-01-24T18:52:32.474 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_service_require_version": "2", 2023-01-24T18:52:32.475 INFO:teuthology.orchestra.run.smithi145.stdout: "cephx_sign_messages": "true", 2023-01-24T18:52:32.475 INFO:teuthology.orchestra.run.smithi145.stdout: "chdir": "", 2023-01-24T18:52:32.475 INFO:teuthology.orchestra.run.smithi145.stdout: "client_acl_type": "", 2023-01-24T18:52:32.475 INFO:teuthology.orchestra.run.smithi145.stdout: "client_asio_thread_count": "2", 2023-01-24T18:52:32.476 INFO:teuthology.orchestra.run.smithi145.stdout: "client_cache_mid": "0.750000", 2023-01-24T18:52:32.476 INFO:teuthology.orchestra.run.smithi145.stdout: "client_cache_size": "16384", 2023-01-24T18:52:32.476 INFO:teuthology.orchestra.run.smithi145.stdout: "client_caps_release_delay": "5", 2023-01-24T18:52:32.476 INFO:teuthology.orchestra.run.smithi145.stdout: "client_check_pool_perm": "true", 2023-01-24T18:52:32.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-24T18:52:32.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_debug_force_sync_read": "false", 2023-01-24T18:52:32.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_debug_getattr_caps": "false", 2023-01-24T18:52:32.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_debug_inject_tick_delay": "0", 2023-01-24T18:52:32.477 INFO:teuthology.orchestra.run.smithi145.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-24T18:52:32.478 INFO:teuthology.orchestra.run.smithi145.stdout: "client_die_on_failed_remount": "false", 2023-01-24T18:52:32.478 INFO:teuthology.orchestra.run.smithi145.stdout: "client_dirsize_rbytes": "true", 2023-01-24T18:52:32.478 INFO:teuthology.orchestra.run.smithi145.stdout: "client_force_lazyio": "false", 2023-01-24T18:52:32.478 INFO:teuthology.orchestra.run.smithi145.stdout: "client_fs": "", 2023-01-24T18:52:32.478 INFO:teuthology.orchestra.run.smithi145.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-24T18:52:32.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_inject_release_failure": "false", 2023-01-24T18:52:32.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_max_inline_size": "4096", 2023-01-24T18:52:32.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_mds_namespace": "", 2023-01-24T18:52:32.479 INFO:teuthology.orchestra.run.smithi145.stdout: "client_metadata": "", 2023-01-24T18:52:32.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_mount_gid": "-1", 2023-01-24T18:52:32.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_mount_timeout": "300", 2023-01-24T18:52:32.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_mount_uid": "-1", 2023-01-24T18:52:32.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_mountpoint": "/", 2023-01-24T18:52:32.480 INFO:teuthology.orchestra.run.smithi145.stdout: "client_notify_timeout": "10", 2023-01-24T18:52:32.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc": "true", 2023-01-24T18:52:32.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc_max_dirty": "104857600", 2023-01-24T18:52:32.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-24T18:52:32.481 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc_max_objects": "1000", 2023-01-24T18:52:32.482 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc_size": "209715200", 2023-01-24T18:52:32.482 INFO:teuthology.orchestra.run.smithi145.stdout: "client_oc_target_dirty": "8388608", 2023-01-24T18:52:32.482 INFO:teuthology.orchestra.run.smithi145.stdout: "client_permissions": "true", 2023-01-24T18:52:32.482 INFO:teuthology.orchestra.run.smithi145.stdout: "client_quota": "true", 2023-01-24T18:52:32.482 INFO:teuthology.orchestra.run.smithi145.stdout: "client_quota_df": "true", 2023-01-24T18:52:32.483 INFO:teuthology.orchestra.run.smithi145.stdout: "client_readahead_max_bytes": "0", 2023-01-24T18:52:32.483 INFO:teuthology.orchestra.run.smithi145.stdout: "client_readahead_max_periods": "4", 2023-01-24T18:52:32.483 INFO:teuthology.orchestra.run.smithi145.stdout: "client_readahead_min": "131072", 2023-01-24T18:52:32.483 INFO:teuthology.orchestra.run.smithi145.stdout: "client_reconnect_stale": "false", 2023-01-24T18:52:32.483 INFO:teuthology.orchestra.run.smithi145.stdout: "client_shutdown_timeout": "30", 2023-01-24T18:52:32.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_snapdir": ".snap", 2023-01-24T18:52:32.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_tick_interval": "1", 2023-01-24T18:52:32.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_trace": "", 2023-01-24T18:52:32.484 INFO:teuthology.orchestra.run.smithi145.stdout: "client_try_dentry_invalidate": "false", 2023-01-24T18:52:32.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_use_faked_inos": "false", 2023-01-24T18:52:32.485 INFO:teuthology.orchestra.run.smithi145.stdout: "client_use_random_mds": "false", 2023-01-24T18:52:32.485 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_graylog": "false", 2023-01-24T18:52:32.485 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-24T18:52:32.485 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_graylog_port": "12201", 2023-01-24T18:52:32.486 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_monitors": "default=true", 2023-01-24T18:52:32.486 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_syslog": "false", 2023-01-24T18:52:32.486 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-24T18:52:32.487 INFO:teuthology.orchestra.run.smithi145.stdout: "clog_to_syslog_level": "info", 2023-01-24T18:52:32.487 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_addr": "-", 2023-01-24T18:52:32.487 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_network": "", 2023-01-24T18:52:32.487 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_network_interface": "", 2023-01-24T18:52:32.488 INFO:teuthology.orchestra.run.smithi145.stdout: "compressor_zlib_isal": "false", 2023-01-24T18:52:32.488 INFO:teuthology.orchestra.run.smithi145.stdout: "compressor_zlib_level": "5", 2023-01-24T18:52:32.488 INFO:teuthology.orchestra.run.smithi145.stdout: "compressor_zlib_winsize": "-15", 2023-01-24T18:52:32.488 INFO:teuthology.orchestra.run.smithi145.stdout: "compressor_zstd_level": "1", 2023-01-24T18:52:32.488 INFO:teuthology.orchestra.run.smithi145.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-24T18:52:32.489 INFO:teuthology.orchestra.run.smithi145.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-24T18:52:32.489 INFO:teuthology.orchestra.run.smithi145.stdout: "crimson_alien_op_num_threads": "6", 2023-01-24T18:52:32.489 INFO:teuthology.orchestra.run.smithi145.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-24T18:52:32.489 INFO:teuthology.orchestra.run.smithi145.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-24T18:52:32.490 INFO:teuthology.orchestra.run.smithi145.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-24T18:52:32.490 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_location": "", 2023-01-24T18:52:32.490 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_location_hook": "", 2023-01-24T18:52:32.490 INFO:teuthology.orchestra.run.smithi145.stdout: "crush_location_hook_timeout": "10", 2023-01-24T18:52:32.490 INFO:teuthology.orchestra.run.smithi145.stdout: "daemonize": "false", 2023-01-24T18:52:32.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_alienstore": "0/5", 2023-01-24T18:52:32.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_allow_any_pool_priority": "false", 2023-01-24T18:52:32.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_asok": "1/5", 2023-01-24T18:52:32.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_asok_assert_abort": "false", 2023-01-24T18:52:32.491 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_asserts_on_shutdown": "true", 2023-01-24T18:52:32.492 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_auth": "20/20", 2023-01-24T18:52:32.492 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_bdev": "1/3", 2023-01-24T18:52:32.492 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_bluefs": "1/5", 2023-01-24T18:52:32.492 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_bluestore": "1/5", 2023-01-24T18:52:32.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_buffer": "0/1", 2023-01-24T18:52:32.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_ceph_exporter": "1/5", 2023-01-24T18:52:32.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_cephfs_mirror": "0/5", 2023-01-24T18:52:32.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_cephsqlite": "0/5", 2023-01-24T18:52:32.493 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_civetweb": "1/10", 2023-01-24T18:52:32.494 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_client": "0/5", 2023-01-24T18:52:32.494 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_compressor": "1/5", 2023-01-24T18:52:32.494 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_context": "0/1", 2023-01-24T18:52:32.494 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_crush": "1/1", 2023-01-24T18:52:32.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_crypto": "1/5", 2023-01-24T18:52:32.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_deliberately_leak_memory": "false", 2023-01-24T18:52:32.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_disable_randomized_ping": "false", 2023-01-24T18:52:32.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_dpdk": "1/5", 2023-01-24T18:52:32.495 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_eventtrace": "1/5", 2023-01-24T18:52:32.496 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_filer": "0/1", 2023-01-24T18:52:32.496 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_filestore": "1/3", 2023-01-24T18:52:32.496 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_finisher": "1/1", 2023-01-24T18:52:32.496 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_fuse": "1/5", 2023-01-24T18:52:32.497 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_heartbeat_testing_span": "0", 2023-01-24T18:52:32.497 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_heartbeatmap": "1/5", 2023-01-24T18:52:32.497 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-24T18:52:32.497 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_javaclient": "1/5", 2023-01-24T18:52:32.497 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_journal": "1/3", 2023-01-24T18:52:32.498 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_journaler": "0/5", 2023-01-24T18:52:32.498 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_kstore": "1/5", 2023-01-24T18:52:32.498 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_leveldb": "4/5", 2023-01-24T18:52:32.498 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_lockdep": "0/1", 2023-01-24T18:52:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mclock": "1/5", 2023-01-24T18:52:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds": "1/5", 2023-01-24T18:52:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds_balancer": "1/5", 2023-01-24T18:52:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds_locker": "1/5", 2023-01-24T18:52:32.499 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds_log": "1/5", 2023-01-24T18:52:32.500 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds_log_expire": "1/5", 2023-01-24T18:52:32.500 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mds_migrator": "1/5", 2023-01-24T18:52:32.500 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_memdb": "4/5", 2023-01-24T18:52:32.500 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mgr": "2/5", 2023-01-24T18:52:32.501 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mgrc": "1/5", 2023-01-24T18:52:32.501 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_mon": "20/20", 2023-01-24T18:52:32.501 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_monc": "0/10", 2023-01-24T18:52:32.501 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_ms": "1/1", 2023-01-24T18:52:32.501 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_none": "0/5", 2023-01-24T18:52:32.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_objclass": "0/5", 2023-01-24T18:52:32.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_objectcacher": "0/5", 2023-01-24T18:52:32.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_objecter": "0/1", 2023-01-24T18:52:32.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_optracker": "0/5", 2023-01-24T18:52:32.502 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_osd": "1/5", 2023-01-24T18:52:32.503 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_paxos": "20/20", 2023-01-24T18:52:32.503 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_perfcounter": "1/5", 2023-01-24T18:52:32.503 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_prioritycache": "1/5", 2023-01-24T18:52:32.503 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rados": "0/5", 2023-01-24T18:52:32.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rbd": "0/5", 2023-01-24T18:52:32.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rbd_mirror": "0/5", 2023-01-24T18:52:32.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rbd_pwl": "0/5", 2023-01-24T18:52:32.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rbd_replay": "0/5", 2023-01-24T18:52:32.504 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_refs": "0/0", 2023-01-24T18:52:32.505 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_reserver": "1/1", 2023-01-24T18:52:32.505 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rgw": "1/5", 2023-01-24T18:52:32.505 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rgw_datacache": "1/5", 2023-01-24T18:52:32.505 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rgw_sync": "1/5", 2023-01-24T18:52:32.506 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_rocksdb": "4/5", 2023-01-24T18:52:32.506 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore": "0/5", 2023-01-24T18:52:32.506 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_cache": "0/5", 2023-01-24T18:52:32.506 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_cleaner": "0/5", 2023-01-24T18:52:32.507 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_device": "0/5", 2023-01-24T18:52:32.507 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_journal": "0/5", 2023-01-24T18:52:32.507 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_lba": "0/5", 2023-01-24T18:52:32.507 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_odata": "0/5", 2023-01-24T18:52:32.507 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_omap": "0/5", 2023-01-24T18:52:32.508 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_onode": "0/5", 2023-01-24T18:52:32.508 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_seastore_tm": "0/5", 2023-01-24T18:52:32.508 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_striper": "0/1", 2023-01-24T18:52:32.508 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_test": "0/5", 2023-01-24T18:52:32.509 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_throttle": "1/1", 2023-01-24T18:52:32.509 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_timer": "0/1", 2023-01-24T18:52:32.509 INFO:teuthology.orchestra.run.smithi145.stdout: "debug_tp": "0/5", 2023-01-24T18:52:32.509 INFO:teuthology.orchestra.run.smithi145.stdout: "device_failure_prediction_mode": "none", 2023-01-24T18:52:32.509 INFO:teuthology.orchestra.run.smithi145.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-24T18:52:32.510 INFO:teuthology.orchestra.run.smithi145.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-24T18:52:32.510 INFO:teuthology.orchestra.run.smithi145.stdout: "err_to_graylog": "false", 2023-01-24T18:52:32.510 INFO:teuthology.orchestra.run.smithi145.stdout: "err_to_journald": "false", 2023-01-24T18:52:32.510 INFO:teuthology.orchestra.run.smithi145.stdout: "err_to_stderr": "true", 2023-01-24T18:52:32.510 INFO:teuthology.orchestra.run.smithi145.stdout: "err_to_syslog": "false", 2023-01-24T18:52:32.511 INFO:teuthology.orchestra.run.smithi145.stdout: "event_tracing": "false", 2023-01-24T18:52:32.511 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_addr": "0.0.0.0", 2023-01-24T18:52:32.511 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_http_port": "9926", 2023-01-24T18:52:32.511 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_prio_limit": "5", 2023-01-24T18:52:32.511 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-24T18:52:32.512 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_sort_metrics": "true", 2023-01-24T18:52:32.512 INFO:teuthology.orchestra.run.smithi145.stdout: "exporter_stats_period": "5", 2023-01-24T18:52:32.512 INFO:teuthology.orchestra.run.smithi145.stdout: "fake_statfs_for_testing": "0", 2023-01-24T18:52:32.512 INFO:teuthology.orchestra.run.smithi145.stdout: "fatal_signal_handlers": "true", 2023-01-24T18:52:32.513 INFO:teuthology.orchestra.run.smithi145.stdout: "filer_max_purge_ops": "10", 2023-01-24T18:52:32.513 INFO:teuthology.orchestra.run.smithi145.stdout: "filer_max_truncate_ops": "128", 2023-01-24T18:52:32.513 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_apply_finisher_threads": "1", 2023-01-24T18:52:32.513 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_blackhole": "false", 2023-01-24T18:52:32.513 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_btrfs_clone_range": "true", 2023-01-24T18:52:32.514 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_btrfs_snap": "true", 2023-01-24T18:52:32.514 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_caller_concurrency": "10", 2023-01-24T18:52:32.514 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_collect_device_partition_information": "true", 2023-01-24T18:52:32.514 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_commit_timeout": "600.000000", 2023-01-24T18:52:32.515 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_debug_inject_read_err": "false", 2023-01-24T18:52:32.515 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_debug_omap_check": "false", 2023-01-24T18:52:32.515 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-24T18:52:32.515 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_debug_verify_split": "false", 2023-01-24T18:52:32.515 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_dump_file": "", 2023-01-24T18:52:32.516 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-24T18:52:32.516 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-24T18:52:32.516 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fadvise": "true", 2023-01-24T18:52:32.516 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fail_eio": "true", 2023-01-24T18:52:32.517 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fd_cache_shards": "16", 2023-01-24T18:52:32.517 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fd_cache_size": "128", 2023-01-24T18:52:32.517 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fiemap": "false", 2023-01-24T18:52:32.517 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fiemap_threshold": "4096", 2023-01-24T18:52:32.517 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-24T18:52:32.518 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-24T18:52:32.518 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_inject_stall": "0", 2023-01-24T18:52:32.518 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_journal_parallel": "false", 2023-01-24T18:52:32.518 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_journal_trailing": "false", 2023-01-24T18:52:32.518 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_journal_writeahead": "false", 2023-01-24T18:52:32.519 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_kill_at": "0", 2023-01-24T18:52:32.519 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-24T18:52:32.519 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-24T18:52:32.519 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-24T18:52:32.520 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-24T18:52:32.520 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-24T18:52:32.520 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattrs": "0", 2023-01-24T18:52:32.520 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-24T18:52:32.520 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-24T18:52:32.521 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-24T18:52:32.521 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-24T18:52:32.521 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_xattr_value_size": "0", 2023-01-24T18:52:32.521 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-24T18:52:32.522 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-24T18:52:32.522 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-24T18:52:32.522 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_merge_threshold": "-10", 2023-01-24T18:52:32.522 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-24T18:52:32.522 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_odsync_write": "false", 2023-01-24T18:52:32.523 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_omap_backend": "rocksdb", 2023-01-24T18:52:32.523 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_omap_backend_path": "", 2023-01-24T18:52:32.523 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-24T18:52:32.523 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-24T18:52:32.523 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-24T18:52:32.524 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_op_thread_timeout": "60", 2023-01-24T18:52:32.524 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_op_threads": "2", 2023-01-24T18:52:32.524 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_punch_hole": "false", 2023-01-24T18:52:32.524 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-24T18:52:32.524 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-24T18:52:32.525 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-24T18:52:32.525 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-24T18:52:32.525 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-24T18:52:32.525 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-24T18:52:32.526 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-24T18:52:32.526 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-24T18:52:32.526 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-24T18:52:32.526 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_queue_max_ops": "50", 2023-01-24T18:52:32.526 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-24T18:52:32.527 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_seek_data_hole": "false", 2023-01-24T18:52:32.527 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_sloppy_crc": "false", 2023-01-24T18:52:32.527 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-24T18:52:32.527 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_splice": "false", 2023-01-24T18:52:32.527 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_split_multiple": "2", 2023-01-24T18:52:32.528 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_split_rand_factor": "20", 2023-01-24T18:52:32.528 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_update_to": "1000", 2023-01-24T18:52:32.528 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-24T18:52:32.528 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-24T18:52:32.529 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-24T18:52:32.529 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-24T18:52:32.529 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-24T18:52:32.529 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-24T18:52:32.529 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_enable": "true", 2023-01-24T18:52:32.530 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-24T18:52:32.530 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-24T18:52:32.530 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-24T18:52:32.530 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-24T18:52:32.530 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-24T18:52:32.531 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-24T18:52:32.531 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_xfs_extsize": "false", 2023-01-24T18:52:32.531 INFO:teuthology.orchestra.run.smithi145.stdout: "filestore_zfs_snap": "false", 2023-01-24T18:52:32.531 INFO:teuthology.orchestra.run.smithi145.stdout: "fio_dir": "/tmp/fio", 2023-01-24T18:52:32.532 INFO:teuthology.orchestra.run.smithi145.stdout: "fsid": "f7057319-7755-43a0-abfd-bc543eb5d39e", 2023-01-24T18:52:32.532 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_allow_other": "true", 2023-01-24T18:52:32.532 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_atomic_o_trunc": "true", 2023-01-24T18:52:32.532 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_big_writes": "true", 2023-01-24T18:52:32.532 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_debug": "false", 2023-01-24T18:52:32.533 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_default_permissions": "false", 2023-01-24T18:52:32.533 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_disable_pagecache": "false", 2023-01-24T18:52:32.533 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_max_write": "0", 2023-01-24T18:52:32.533 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_multithreaded": "true", 2023-01-24T18:52:32.533 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_require_active_mds": "true", 2023-01-24T18:52:32.534 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_set_user_groups": "true", 2023-01-24T18:52:32.534 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_splice_move": "true", 2023-01-24T18:52:32.534 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_splice_read": "true", 2023-01-24T18:52:32.534 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_splice_write": "true", 2023-01-24T18:52:32.534 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-24T18:52:32.535 INFO:teuthology.orchestra.run.smithi145.stdout: "fuse_use_invalidate_cb": "true", 2023-01-24T18:52:32.535 INFO:teuthology.orchestra.run.smithi145.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-24T18:52:32.535 INFO:teuthology.orchestra.run.smithi145.stdout: "gss_target_name": "ceph", 2023-01-24T18:52:32.535 INFO:teuthology.orchestra.run.smithi145.stdout: "heartbeat_file": "", 2023-01-24T18:52:32.536 INFO:teuthology.orchestra.run.smithi145.stdout: "heartbeat_inject_failure": "0", 2023-01-24T18:52:32.536 INFO:teuthology.orchestra.run.smithi145.stdout: "heartbeat_interval": "5", 2023-01-24T18:52:32.536 INFO:teuthology.orchestra.run.smithi145.stdout: "host": "smithi145", 2023-01-24T18:52:32.536 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-24T18:52:32.536 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-24T18:52:32.537 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-24T18:52:32.537 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-24T18:52:32.537 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-24T18:52:32.537 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-24T18:52:32.538 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-24T18:52:32.538 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-24T18:52:32.538 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-24T18:52:32.538 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-24T18:52:32.538 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-24T18:52:32.539 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-24T18:52:32.539 INFO:teuthology.orchestra.run.smithi145.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-24T18:52:32.539 INFO:teuthology.orchestra.run.smithi145.stdout: "inject_early_sigterm": "false", 2023-01-24T18:52:32.539 INFO:teuthology.orchestra.run.smithi145.stdout: "jaeger_tracing_enable": "false", 2023-01-24T18:52:32.539 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_aio": "true", 2023-01-24T18:52:32.540 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_align_min_size": "65536", 2023-01-24T18:52:32.540 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_block_align": "true", 2023-01-24T18:52:32.540 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_block_size": "4096", 2023-01-24T18:52:32.540 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_dio": "true", 2023-01-24T18:52:32.540 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_discard": "false", 2023-01-24T18:52:32.541 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_force_aio": "false", 2023-01-24T18:52:32.541 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_ignore_corruption": "false", 2023-01-24T18:52:32.541 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_max_write_bytes": "10485760", 2023-01-24T18:52:32.541 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_max_write_entries": "100", 2023-01-24T18:52:32.541 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_replay_from": "0", 2023-01-24T18:52:32.542 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-24T18:52:32.542 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-24T18:52:32.542 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-24T18:52:32.542 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-24T18:52:32.542 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_write_header_frequency": "0", 2023-01-24T18:52:32.543 INFO:teuthology.orchestra.run.smithi145.stdout: "journal_zero_on_create": "false", 2023-01-24T18:52:32.543 INFO:teuthology.orchestra.run.smithi145.stdout: "journaler_prefetch_periods": "10", 2023-01-24T18:52:32.543 INFO:teuthology.orchestra.run.smithi145.stdout: "journaler_prezero_periods": "5", 2023-01-24T18:52:32.543 INFO:teuthology.orchestra.run.smithi145.stdout: "journaler_write_head_interval": "15", 2023-01-24T18:52:32.544 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "", 2023-01-24T18:52:32.544 INFO:teuthology.orchestra.run.smithi145.stdout: "keyfile": "", 2023-01-24T18:52:32.544 INFO:teuthology.orchestra.run.smithi145.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-24T18:52:32.544 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_backend": "rocksdb", 2023-01-24T18:52:32.544 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_default_stripe_size": "65536", 2023-01-24T18:52:32.545 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_fsck_on_mount": "false", 2023-01-24T18:52:32.545 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-24T18:52:32.545 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_max_bytes": "67108864", 2023-01-24T18:52:32.545 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_max_ops": "512", 2023-01-24T18:52:32.545 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_nid_prealloc": "1024", 2023-01-24T18:52:32.546 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_onode_map_size": "1024", 2023-01-24T18:52:32.546 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-24T18:52:32.546 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_sync_submit_transaction": "false", 2023-01-24T18:52:32.546 INFO:teuthology.orchestra.run.smithi145.stdout: "kstore_sync_transaction": "false", 2023-01-24T18:52:32.547 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_block_size": "65536", 2023-01-24T18:52:32.547 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_bloom_size": "0", 2023-01-24T18:52:32.547 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_cache_size": "536870912", 2023-01-24T18:52:32.547 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_compact_on_mount": "false", 2023-01-24T18:52:32.547 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_compression": "false", 2023-01-24T18:52:32.548 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_log": "", 2023-01-24T18:52:32.548 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-24T18:52:32.548 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_max_open_files": "0", 2023-01-24T18:52:32.548 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_paranoid": "false", 2023-01-24T18:52:32.548 INFO:teuthology.orchestra.run.smithi145.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-24T18:52:32.549 INFO:teuthology.orchestra.run.smithi145.stdout: "librados_thread_count": "2", 2023-01-24T18:52:32.549 INFO:teuthology.orchestra.run.smithi145.stdout: "lockdep": "false", 2023-01-24T18:52:32.549 INFO:teuthology.orchestra.run.smithi145.stdout: "lockdep_force_backtrace": "false", 2023-01-24T18:52:32.549 INFO:teuthology.orchestra.run.smithi145.stdout: "log_coarse_timestamps": "true", 2023-01-24T18:52:32.550 INFO:teuthology.orchestra.run.smithi145.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-24T18:52:32.550 INFO:teuthology.orchestra.run.smithi145.stdout: "log_flush_on_exit": "false", 2023-01-24T18:52:32.550 INFO:teuthology.orchestra.run.smithi145.stdout: "log_graylog_host": "127.0.0.1", 2023-01-24T18:52:32.550 INFO:teuthology.orchestra.run.smithi145.stdout: "log_graylog_port": "12201", 2023-01-24T18:52:32.550 INFO:teuthology.orchestra.run.smithi145.stdout: "log_max_new": "1000", 2023-01-24T18:52:32.551 INFO:teuthology.orchestra.run.smithi145.stdout: "log_max_recent": "10000", 2023-01-24T18:52:32.551 INFO:teuthology.orchestra.run.smithi145.stdout: "log_stderr_prefix": "", 2023-01-24T18:52:32.551 INFO:teuthology.orchestra.run.smithi145.stdout: "log_stop_at_utilization": "0.970000", 2023-01-24T18:52:32.551 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_file": "true", 2023-01-24T18:52:32.551 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_graylog": "false", 2023-01-24T18:52:32.552 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_journald": "false", 2023-01-24T18:52:32.552 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_stderr": "false", 2023-01-24T18:52:32.552 INFO:teuthology.orchestra.run.smithi145.stdout: "log_to_syslog": "false", 2023-01-24T18:52:32.552 INFO:teuthology.orchestra.run.smithi145.stdout: "max_rotating_auth_attempts": "10", 2023-01-24T18:52:32.553 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_action_on_write_error": "1", 2023-01-24T18:52:32.553 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_alternate_name_max": "8192", 2023-01-24T18:52:32.553 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_asio_thread_count": "2", 2023-01-24T18:52:32.553 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_export_pin": "true", 2023-01-24T18:52:32.553 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_fragment_dirs": "true", 2023-01-24T18:52:32.554 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-24T18:52:32.554 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_fragment_interval": "5", 2023-01-24T18:52:32.554 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-24T18:52:32.554 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-24T18:52:32.554 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_interval": "10", 2023-01-24T18:52:32.555 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_max": "-1", 2023-01-24T18:52:32.555 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_max_until": "-1", 2023-01-24T18:52:32.555 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_merge_size": "50", 2023-01-24T18:52:32.555 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_midchunk": "0.300000", 2023-01-24T18:52:32.556 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-24T18:52:32.556 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_min_start": "0.200000", 2023-01-24T18:52:32.556 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_minchunk": "0.001000", 2023-01-24T18:52:32.556 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_mode": "0", 2023-01-24T18:52:32.557 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_need_max": "1.200000", 2023-01-24T18:52:32.557 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_need_min": "0.800000", 2023-01-24T18:52:32.557 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-24T18:52:32.557 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-24T18:52:32.557 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_split_bits": "3", 2023-01-24T18:52:32.558 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-24T18:52:32.558 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_split_size": "10000", 2023-01-24T18:52:32.558 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-24T18:52:32.558 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_target_decay": "10.000000", 2023-01-24T18:52:32.558 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-24T18:52:32.559 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_beacon_grace": "15.000000", 2023-01-24T18:52:32.559 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_beacon_interval": "4.000000", 2023-01-24T18:52:32.559 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-24T18:52:32.559 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-24T18:52:32.559 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_mid": "0.700000", 2023-01-24T18:52:32.560 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_release_free_interval": "10", 2023-01-24T18:52:32.560 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_reservation": "0.050000", 2023-01-24T18:52:32.560 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-24T18:52:32.560 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_trim_interval": "1", 2023-01-24T18:52:32.561 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache_trim_threshold": "262144", 2023-01-24T18:52:32.561 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-24T18:52:32.561 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-24T18:52:32.561 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-24T18:52:32.562 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_client_prealloc_inos": "1000", 2023-01-24T18:52:32.562 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-24T18:52:32.562 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_connect_bootstrapping": "false", 2023-01-24T18:52:32.562 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_damage_table_max_entries": "10000", 2023-01-24T18:52:32.562 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-24T18:52:32.563 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_debug_auth_pins": "false", 2023-01-24T18:52:32.563 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_debug_frag": "false", 2023-01-24T18:52:32.563 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_debug_scatterstat": "false", 2023-01-24T18:52:32.563 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_debug_subtrees": "false", 2023-01-24T18:52:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_decay_halflife": "5.000000", 2023-01-24T18:52:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_default_dir_hash": "2", 2023-01-24T18:52:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_defer_session_stale": "true", 2023-01-24T18:52:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_deny_all_reconnect": "false", 2023-01-24T18:52:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dir_keys_per_op": "16384", 2023-01-24T18:52:32.565 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dir_max_commit_size": "10", 2023-01-24T18:52:32.565 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dir_max_entries": "0", 2023-01-24T18:52:32.565 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-24T18:52:32.565 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-24T18:52:32.566 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dump_cache_on_map": "false", 2023-01-24T18:52:32.566 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-24T18:52:32.566 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-24T18:52:32.566 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_early_reply": "true", 2023-01-24T18:52:32.566 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_enable_op_tracker": "true", 2023-01-24T18:52:32.567 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_enforce_unique_name": "true", 2023-01-24T18:52:32.567 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-24T18:52:32.567 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-24T18:52:32.567 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_export_ephemeral_random": "true", 2023-01-24T18:52:32.567 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-24T18:52:32.568 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-24T18:52:32.568 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-24T18:52:32.568 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-24T18:52:32.568 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-24T18:52:32.569 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_health_summarize_threshold": "10", 2023-01-24T18:52:32.569 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-24T18:52:32.569 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-24T18:52:32.569 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_inject_health_dummy": "false", 2023-01-24T18:52:32.569 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_inject_migrator_session_race": "false", 2023-01-24T18:52:32.570 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-24T18:52:32.570 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_join_fs": "", 2023-01-24T18:52:32.570 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_journal_format": "1", 2023-01-24T18:52:32.570 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_create_at": "0", 2023-01-24T18:52:32.570 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_export_at": "0", 2023-01-24T18:52:32.571 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_import_at": "0", 2023-01-24T18:52:32.571 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_journal_at": "0", 2023-01-24T18:52:32.571 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_journal_expire_at": "0", 2023-01-24T18:52:32.571 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_journal_replay_at": "0", 2023-01-24T18:52:32.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_link_at": "0", 2023-01-24T18:52:32.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_mdstable_at": "0", 2023-01-24T18:52:32.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_openc_at": "0", 2023-01-24T18:52:32.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_kill_rename_at": "0", 2023-01-24T18:52:32.572 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_events_per_segment": "1024", 2023-01-24T18:52:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_max_events": "-1", 2023-01-24T18:52:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_max_segments": "128", 2023-01-24T18:52:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_pause": "false", 2023-01-24T18:52:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_segment_size": "0", 2023-01-24T18:52:32.573 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-24T18:52:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log_warn_factor": "2.000000", 2023-01-24T18:52:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_caps_per_client": "1048576", 2023-01-24T18:52:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_completed_flushes": "100000", 2023-01-24T18:52:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_completed_requests": "100000", 2023-01-24T18:52:32.574 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_export_size": "20971520", 2023-01-24T18:52:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_file_recover": "32", 2023-01-24T18:52:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_purge_files": "64", 2023-01-24T18:52:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_purge_ops": "8192", 2023-01-24T18:52:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-24T18:52:32.575 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-24T18:52:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-24T18:52:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_snaps_per_dir": "100", 2023-01-24T18:52:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-24T18:52:32.576 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_metrics_update_interval": "2", 2023-01-24T18:52:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_min_caps_per_client": "100", 2023-01-24T18:52:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_min_caps_working_set": "10000", 2023-01-24T18:52:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-24T18:52:32.577 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_numa_node": "-1", 2023-01-24T18:52:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-24T18:52:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_op_complaint_time": "30.000000", 2023-01-24T18:52:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_op_history_duration": "600", 2023-01-24T18:52:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_op_history_size": "20", 2023-01-24T18:52:32.578 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_op_log_threshold": "5", 2023-01-24T18:52:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_ping_grace": "15", 2023-01-24T18:52:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_ping_interval": "5", 2023-01-24T18:52:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-24T18:52:32.579 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-24T18:52:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_max_caps": "30000", 2023-01-24T18:52:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-24T18:52:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-24T18:52:32.580 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-24T18:52:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_recall_warning_threshold": "262144", 2023-01-24T18:52:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-24T18:52:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_replay_interval": "1.000000", 2023-01-24T18:52:32.581 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-24T18:52:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-24T18:52:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_root_ino_gid": "0", 2023-01-24T18:52:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_root_ino_uid": "0", 2023-01-24T18:52:32.582 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-24T18:52:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-24T18:52:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-24T18:52:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-24T18:52:32.583 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-24T18:52:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-24T18:52:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-24T18:52:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-24T18:52:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-24T18:52:32.584 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_shutdown_check": "0", 2023-01-24T18:52:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_skip_ino": "0", 2023-01-24T18:52:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-24T18:52:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_snap_max_uid": "4294967294", 2023-01-24T18:52:32.585 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_snap_min_uid": "0", 2023-01-24T18:52:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_snap_rstat": "false", 2023-01-24T18:52:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_standby_replay_damaged": "false", 2023-01-24T18:52:32.586 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_symlink_recovery": "true", 2023-01-24T18:52:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-24T18:52:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_thrash_exports": "0", 2023-01-24T18:52:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_thrash_fragments": "0", 2023-01-24T18:52:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_tick_interval": "5.000000", 2023-01-24T18:52:32.587 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_valgrind_exit": "false", 2023-01-24T18:52:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_verify_backtrace": "1", 2023-01-24T18:52:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_verify_scatter": "false", 2023-01-24T18:52:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-24T18:52:32.588 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_wipe_sessions": "false", 2023-01-24T18:52:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool_debug": "false", 2023-01-24T18:52:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-24T18:52:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: "memstore_device_bytes": "1073741824", 2023-01-24T18:52:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: "memstore_page_set": "false", 2023-01-24T18:52:32.589 INFO:teuthology.orchestra.run.smithi145.stdout: "memstore_page_size": "65536", 2023-01-24T18:52:32.590 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_client_bytes": "134217728", 2023-01-24T18:52:32.590 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_client_messages": "512", 2023-01-24T18:52:32.590 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-24T18:52:32.590 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-24T18:52:32.591 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-24T18:52:32.591 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-24T18:52:32.591 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_disabled_modules": "", 2023-01-24T18:52:32.591 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-24T18:52:32.591 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_max_pg_num_change": "128", 2023-01-24T18:52:32.592 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_mds_bytes": "134217728", 2023-01-24T18:52:32.592 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_mds_messages": "128", 2023-01-24T18:52:32.592 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-24T18:52:32.592 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_mon_bytes": "134217728", 2023-01-24T18:52:32.592 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_mon_messages": "128", 2023-01-24T18:52:32.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_osd_bytes": "536870912", 2023-01-24T18:52:32.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_osd_messages": "8192", 2023-01-24T18:52:32.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_pool": "true", 2023-01-24T18:52:32.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-24T18:52:32.593 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_standby_modules": "true", 2023-01-24T18:52:32.594 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_stats_period": "5", 2023-01-24T18:52:32.594 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_stats_threshold": "5", 2023-01-24T18:52:32.594 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_tick_period": "2", 2023-01-24T18:52:32.594 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-24T18:52:32.594 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-24T18:52:32.595 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_allow_pool_delete": "true", 2023-01-24T18:52:32.595 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_allow_pool_size_one": "true", 2023-01-24T18:52:32.595 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_auth_validate_all_caps": "true", 2023-01-24T18:52:32.595 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-24T18:52:32.595 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-24T18:52:32.596 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_bytes": "104857600", 2023-01-24T18:52:32.596 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_directed_command_retry": "2", 2023-01-24T18:52:32.596 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-24T18:52:32.596 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-24T18:52:32.597 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-24T18:52:32.597 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-24T18:52:32.597 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_hunt_parallel": "3", 2023-01-24T18:52:32.597 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_log_interval": "1.000000", 2023-01-24T18:52:32.597 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-24T18:52:32.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_ping_interval": "10.000000", 2023-01-24T18:52:32.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-24T18:52:32.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-24T18:52:32.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-24T18:52:32.598 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-24T18:52:32.599 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_file_level": "debug", 2023-01-24T18:52:32.599 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_file": "true", 2023-01-24T18:52:32.599 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-24T18:52:32.599 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-24T18:52:32.600 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-24T18:52:32.600 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_journald": "false", 2023-01-24T18:52:32.600 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-24T18:52:32.600 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-24T18:52:32.600 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-24T18:52:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-24T18:52:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_compact_on_bootstrap": "false", 2023-01-24T18:52:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_compact_on_start": "false", 2023-01-24T18:52:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_compact_on_trim": "true", 2023-01-24T18:52:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-24T18:52:32.602 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-24T18:52:32.602 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-24T18:52:32.602 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_cpu_threads": "4", 2023-01-24T18:52:32.602 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_crush_min_required_version": "hammer", 2023-01-24T18:52:32.603 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_daemon_bytes": "419430400", 2023-01-24T18:52:32.603 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-24T18:52:32.603 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_data_avail_crit": "5", 2023-01-24T18:52:32.603 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_data_avail_warn": "5", 2023-01-24T18:52:32.604 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_data_size_warn": "16106127360", 2023-01-24T18:52:32.604 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-24T18:52:32.604 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-24T18:52:32.605 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_dump_json": "false", 2023-01-24T18:52:32.605 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-24T18:52:32.605 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_dump_transactions": "false", 2023-01-24T18:52:32.605 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_extra_checks": "false", 2023-01-24T18:52:32.606 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-24T18:52:32.606 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-24T18:52:32.606 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_no_require_pacific": "false", 2023-01-24T18:52:32.606 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_no_require_quincy": "false", 2023-01-24T18:52:32.607 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-24T18:52:32.607 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-24T18:52:32.607 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-24T18:52:32.607 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_down_mkfs_grace": "120", 2023-01-24T18:52:32.607 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_election_default_strategy": "1", 2023-01-24T18:52:32.608 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_election_timeout": "5.000000", 2023-01-24T18:52:32.608 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-24T18:52:32.608 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_elector_ping_divisor": "2", 2023-01-24T18:52:32.608 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-24T18:52:32.608 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_enable_op_tracker": "true", 2023-01-24T18:52:32.609 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_fake_pool_delete": "false", 2023-01-24T18:52:32.609 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_force_quorum_join": "false", 2023-01-24T18:52:32.609 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_globalid_prealloc": "10000", 2023-01-24T18:52:32.609 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_detail_to_clog": "false", 2023-01-24T18:52:32.609 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_log_update_period": "5", 2023-01-24T18:52:32.610 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_max_detail": "50", 2023-01-24T18:52:32.610 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_to_clog": "true", 2023-01-24T18:52:32.610 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_to_clog_interval": "600", 2023-01-24T18:52:32.610 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-24T18:52:32.610 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_host": "172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790],172.21.15.167", 2023-01-24T18:52:32.611 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_host_override": "", 2023-01-24T18:52:32.611 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_initial_members": "", 2023-01-24T18:52:32.611 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-24T18:52:32.611 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-24T18:52:32.611 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-24T18:52:32.612 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-24T18:52:32.612 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-24T18:52:32.612 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_lease": "5.000000", 2023-01-24T18:52:32.612 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-24T18:52:32.613 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-24T18:52:32.613 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_log_full_interval": "50", 2023-01-24T18:52:32.613 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_log_max": "10000", 2023-01-24T18:52:32.613 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_log_max_summary": "50", 2023-01-24T18:52:32.613 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-24T18:52:32.614 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_log_epochs": "500", 2023-01-24T18:52:32.614 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-24T18:52:32.614 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-24T18:52:32.614 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_osd": "10000", 2023-01-24T18:52:32.614 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_pg_per_osd": "10000", 2023-01-24T18:52:32.615 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_pool_pg_num": "65536", 2023-01-24T18:52:32.615 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-24T18:52:32.615 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-24T18:52:32.615 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mds_force_trim_to": "0", 2023-01-24T18:52:32.615 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mds_skip_sanity": "false", 2023-01-24T18:52:32.616 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_memory_autotune": "true", 2023-01-24T18:52:32.616 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_memory_target": "2147483648", 2023-01-24T18:52:32.616 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_beacon_grace": "30", 2023-01-24T18:52:32.616 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-24T18:52:32.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_digest_period": "5", 2023-01-24T18:52:32.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_inactive_grace": "60", 2023-01-24T18:52:32.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-24T18:52:32.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-24T18:52:32.617 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_min_osdmap_epochs": "50", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_complaint_time": "30", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_history_duration": "600", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_history_size": "20", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_history_slow_op_size": "20", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-24T18:52:32.618 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_op_log_threshold": "5", 2023-01-24T18:52:32.619 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-24T18:52:32.619 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-24T18:52:32.619 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-24T18:52:32.619 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_auto_mark_in": "false", 2023-01-24T18:52:32.620 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-24T18:52:32.620 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-24T18:52:32.620 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-24T18:52:32.620 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_cache_size": "500", 2023-01-24T18:52:32.620 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-24T18:52:32.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-24T18:52:32.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-24T18:52:32.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_down_out_interval": "600", 2023-01-24T18:52:32.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-24T18:52:32.621 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-24T18:52:32.622 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_force_trim_to": "0", 2023-01-24T18:52:32.622 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-24T18:52:32.622 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-24T18:52:32.622 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-24T18:52:32.623 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-24T18:52:32.623 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-24T18:52:32.623 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-24T18:52:32.623 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-24T18:52:32.623 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-24T18:52:32.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_min_down_reporters": "2", 2023-01-24T18:52:32.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-24T18:52:32.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-24T18:52:32.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-24T18:52:32.624 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-24T18:52:32.625 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-24T18:52:32.625 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-24T18:52:32.625 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_report_timeout": "900", 2023-01-24T18:52:32.625 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-24T18:52:32.625 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-24T18:52:32.626 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-24T18:52:32.626 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-24T18:52:32.626 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-24T18:52:32.626 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-24T18:52:32.626 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-24T18:52:32.627 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-24T18:52:32.627 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-24T18:52:32.627 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_stuck_threshold": "60", 2023-01-24T18:52:32.627 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-24T18:52:32.627 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-24T18:52:32.628 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-24T18:52:32.628 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-24T18:52:32.628 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-24T18:52:32.628 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-24T18:52:32.629 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_probe_timeout": "2.000000", 2023-01-24T18:52:32.629 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_reweight_max_change": "0.050000", 2023-01-24T18:52:32.629 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_reweight_max_osds": "4", 2023-01-24T18:52:32.629 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-24T18:52:32.629 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-24T18:52:32.630 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-24T18:52:32.630 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-24T18:52:32.630 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-24T18:52:32.630 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_scrub_interval": "300", 2023-01-24T18:52:32.630 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_scrub_max_keys": "100", 2023-01-24T18:52:32.631 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_scrub_timeout": "300", 2023-01-24T18:52:32.631 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_session_timeout": "300", 2023-01-24T18:52:32.631 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_smart_report_timeout": "5", 2023-01-24T18:52:32.631 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_stat_smooth_intervals": "6", 2023-01-24T18:52:32.631 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-24T18:52:32.632 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_stretch_pool_min_size": "2", 2023-01-24T18:52:32.632 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_stretch_pool_size": "4", 2023-01-24T18:52:32.632 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-24T18:52:32.632 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-24T18:52:32.632 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_debug": "false", 2023-01-24T18:52:32.633 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-24T18:52:32.633 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-24T18:52:32.633 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_provider_kill_at": "0", 2023-01-24T18:52:32.633 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_requester_kill_at": "0", 2023-01-24T18:52:32.634 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_sync_timeout": "60.000000", 2023-01-24T18:52:32.634 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_target_pg_per_osd": "100", 2023-01-24T18:52:32.634 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_tick_interval": "5", 2023-01-24T18:52:32.634 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_timecheck_interval": "300.000000", 2023-01-24T18:52:32.634 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-24T18:52:32.635 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_older_version_delay": "604800", 2023-01-24T18:52:32.635 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-24T18:52:32.635 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-24T18:52:32.635 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-24T18:52:32.635 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-24T18:52:32.636 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-24T18:52:32.636 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-24T18:52:32.636 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-24T18:52:32.636 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_misplaced": "false", 2023-01-24T18:52:32.637 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-24T18:52:32.637 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_older_version": "true", 2023-01-24T18:52:32.637 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-24T18:52:32.637 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-24T18:52:32.637 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-24T18:52:32.638 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-24T18:52:32.638 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-24T18:52:32.638 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-24T18:52:32.638 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-24T18:52:32.638 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-24T18:52:32.639 INFO:teuthology.orchestra.run.smithi145.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-24T18:52:32.639 INFO:teuthology.orchestra.run.smithi145.stdout: "monmap": "", 2023-01-24T18:52:32.639 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_op_threads": "3", 2023-01-24T18:52:32.639 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-24T18:52:32.639 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_cm": "false", 2023-01-24T18:52:32.640 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_device_name": "", 2023-01-24T18:52:32.640 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_dscp": "96", 2023-01-24T18:52:32.640 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-24T18:52:32.640 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-24T18:52:32.640 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_local_gid": "", 2023-01-24T18:52:32.641 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-24T18:52:32.641 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_port_num": "1", 2023-01-24T18:52:32.641 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-24T18:52:32.641 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-24T18:52:32.642 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_roce_ver": "1", 2023-01-24T18:52:32.642 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_send_buffers": "1024", 2023-01-24T18:52:32.642 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_sl": "3", 2023-01-24T18:52:32.642 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_support_srq": "true", 2023-01-24T18:52:32.642 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_rdma_type": "ib", 2023-01-24T18:52:32.643 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_async_reap_threshold": "5", 2023-01-24T18:52:32.643 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_before_connect": "false", 2023-01-24T18:52:32.643 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_ipv4": "true", 2023-01-24T18:52:32.643 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_ipv6": "false", 2023-01-24T18:52:32.643 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_msgr1": "true", 2023-01-24T18:52:32.644 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_msgr2": "true", 2023-01-24T18:52:32.644 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_port_max": "7568", 2023-01-24T18:52:32.644 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_port_min": "6800", 2023-01-24T18:52:32.644 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_prefer_ipv4": "false", 2023-01-24T18:52:32.644 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_retry_count": "3", 2023-01-24T18:52:32.645 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_bind_retry_delay": "5", 2023-01-24T18:52:32.645 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_blackhole_client": "false", 2023-01-24T18:52:32.645 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_blackhole_mds": "false", 2023-01-24T18:52:32.645 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_blackhole_mgr": "false", 2023-01-24T18:52:32.645 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_blackhole_mon": "false", 2023-01-24T18:52:32.646 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_blackhole_osd": "false", 2023-01-24T18:52:32.646 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_client_mode": "crc secure", 2023-01-24T18:52:32.646 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_cluster_mode": "crc secure", 2023-01-24T18:52:32.646 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_cluster_type": "", 2023-01-24T18:52:32.647 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_compress_secure": "false", 2023-01-24T18:52:32.647 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_connection_idle_timeout": "900", 2023-01-24T18:52:32.647 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_connection_ready_timeout": "10", 2023-01-24T18:52:32.648 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_crc_data": "true", 2023-01-24T18:52:32.648 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_crc_header": "true", 2023-01-24T18:52:32.648 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_die_on_bad_msg": "false", 2023-01-24T18:52:32.649 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_die_on_bug": "true", 2023-01-24T18:52:32.649 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_die_on_old_message": "true", 2023-01-24T18:52:32.649 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_die_on_skipped_message": "false", 2023-01-24T18:52:32.649 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_die_on_unhandled_msg": "false", 2023-01-24T18:52:32.649 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-01-24T18:52:32.650 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_coremask": "0xF", 2023-01-24T18:52:32.650 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-01-24T18:52:32.650 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_devs_allowlist": "", 2023-01-24T18:52:32.650 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_enable_tso": "true", 2023-01-24T18:52:32.651 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-01-24T18:52:32.651 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-01-24T18:52:32.651 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_hugepages": "", 2023-01-24T18:52:32.651 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_hw_flow_control": "true", 2023-01-24T18:52:32.651 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-01-24T18:52:32.652 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_lro": "true", 2023-01-24T18:52:32.652 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_memory_channel": "4", 2023-01-24T18:52:32.652 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-01-24T18:52:32.652 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_pmd": "", 2023-01-24T18:52:32.652 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_port_id": "0", 2023-01-24T18:52:32.653 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-01-24T18:52:32.653 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dump_corrupt_message_level": "1", 2023-01-24T18:52:32.653 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_dump_on_send": "false", 2023-01-24T18:52:32.653 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_initial_backoff": "0.200000", 2023-01-24T18:52:32.653 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_inject_delay_max": "1.000000", 2023-01-24T18:52:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_inject_delay_probability": "0.000000", 2023-01-24T18:52:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_inject_delay_type": "", 2023-01-24T18:52:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_inject_internal_delays": "0.000000", 2023-01-24T18:52:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_inject_socket_failures": "0", 2023-01-24T18:52:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_learn_addr_from_peer": "true", 2023-01-24T18:52:32.655 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_max_accept_failures": "4", 2023-01-24T18:52:32.655 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_max_backoff": "15.000000", 2023-01-24T18:52:32.655 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_mon_client_mode": "secure crc", 2023-01-24T18:52:32.655 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_mon_cluster_mode": "secure crc", 2023-01-24T18:52:32.656 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_mon_service_mode": "secure crc", 2023-01-24T18:52:32.656 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_osd_compress_min_size": "1024", 2023-01-24T18:52:32.656 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_osd_compress_mode": "none", 2023-01-24T18:52:32.656 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2023-01-24T18:52:32.656 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-01-24T18:52:32.657 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_pq_min_cost": "65536", 2023-01-24T18:52:32.657 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_public_type": "", 2023-01-24T18:52:32.657 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_service_mode": "crc secure", 2023-01-24T18:52:32.657 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_tcp_listen_backlog": "512", 2023-01-24T18:52:32.657 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_tcp_nodelay": "true", 2023-01-24T18:52:32.658 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-01-24T18:52:32.658 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_tcp_rcvbuf": "0", 2023-01-24T18:52:32.658 INFO:teuthology.orchestra.run.smithi145.stdout: "ms_type": "async", 2023-01-24T18:52:32.658 INFO:teuthology.orchestra.run.smithi145.stdout: "no_config_file": "false", 2023-01-24T18:52:32.659 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_completion_locks_per_session": "32", 2023-01-24T18:52:32.659 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_debug_inject_relock_delay": "false", 2023-01-24T18:52:32.659 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_inflight_op_bytes": "104857600", 2023-01-24T18:52:32.659 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_inflight_ops": "1024", 2023-01-24T18:52:32.659 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_inject_no_watch_ping": "false", 2023-01-24T18:52:32.660 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-01-24T18:52:32.660 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_tick_interval": "5.000000", 2023-01-24T18:52:32.660 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter_timeout": "10.000000", 2023-01-24T18:52:32.660 INFO:teuthology.orchestra.run.smithi145.stdout: "objectstore_blackhole": "false", 2023-01-24T18:52:32.660 INFO:teuthology.orchestra.run.smithi145.stdout: "openssl_engine_opts": "", 2023-01-24T18:52:32.661 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_delay_time": "5.000000", 2023-01-24T18:52:32.661 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_hist_halflife": "1000", 2023-01-24T18:52:32.661 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_max_low_ops": "2", 2023-01-24T18:52:32.661 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_max_ops": "4", 2023-01-24T18:52:32.661 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_quantize_effort": "0.100000", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_agent_slop": "0.020000", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_allow_recovery_below_min_size": "true", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_asio_thread_count": "2", 2023-01-24T18:52:32.662 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_async_recovery_min_cost": "100", 2023-01-24T18:52:32.663 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_auto_mark_unfound_lost": "false", 2023-01-24T18:52:32.663 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backfill_retry_interval": "30.000000", 2023-01-24T18:52:32.663 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backfill_scan_max": "512", 2023-01-24T18:52:32.663 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backfill_scan_min": "64", 2023-01-24T18:52:32.663 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backoff_on_degraded": "false", 2023-01-24T18:52:32.664 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backoff_on_peering": "false", 2023-01-24T18:52:32.664 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_backoff_on_unfound": "true", 2023-01-24T18:52:32.664 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_beacon_report_interval": "300", 2023-01-24T18:52:32.664 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bench_duration": "30", 2023-01-24T18:52:32.664 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-01-24T18:52:32.665 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bench_max_block_size": "67108864", 2023-01-24T18:52:32.665 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bench_small_size_max_iops": "100", 2023-01-24T18:52:32.665 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_blkin_trace_all": "false", 2023-01-24T18:52:32.665 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-01-24T18:52:32.665 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-01-24T18:52:32.666 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_check_for_log_corruption": "false", 2023-01-24T18:52:32.666 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-01-24T18:52:32.666 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T18:52:32.666 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_class_update_on_start": "true", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_client_message_cap": "256", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_client_message_size_cap": "524288000", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_client_op_priority": "63", 2023-01-24T18:52:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_client_watch_timeout": "30", 2023-01-24T18:52:32.668 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_command_max_records": "256", 2023-01-24T18:52:32.668 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_compact_on_start": "false", 2023-01-24T18:52:32.668 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-01-24T18:52:32.668 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_crush_chooseleaf_type": "0", 2023-01-24T18:52:32.669 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_crush_initial_weight": "-1.000000", 2023-01-24T18:52:32.669 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_crush_update_on_start": "true", 2023-01-24T18:52:32.669 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_crush_update_weight_set": "true", 2023-01-24T18:52:32.669 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-01-24T18:52:32.669 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_drop_ping_duration": "0", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_feed_pullee": "-1", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-01-24T18:52:32.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-01-24T18:52:32.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_inject_dispatch_delay_probability": "0.100000", 2023-01-24T18:52:32.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_misdirected_ops": "false", 2023-01-24T18:52:32.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_no_acting_change": "false", 2023-01-24T18:52:32.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_no_purge_strays": "false", 2023-01-24T18:52:32.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_op_order": "false", 2023-01-24T18:52:32.672 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_pg_log_writeout": "false", 2023-01-24T18:52:32.672 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_debug_pretend_recovery_active": "false", 2023-01-24T18:52:32.672 INFO:teutholog