2023-01-24T21:55:57.167 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T21:55:57.168 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870 --verbose -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870/orig.config.yaml 2023-01-24T21:55:57.227 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T21:55:57.309 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} email: yweinstein@radhat.com first_in_suite: false job_id: '7135870' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 - volumes: size: 50 - machine: ram: 15000 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 2500 ms tcp read timeout: 5 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_object_clean_region_max_num_intervals: 1000 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore queue throttle high multiple: 2 filestore queue throttle max multiple: 10 journal throttle high multiple: 2 journal throttle max multiple: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 9 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate mon_bind_msgr2: false sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXB8lkoP5CVWO87fZakSfTl3CmucUtKV6//WjBPrCsmqczBhaoxe3wq87zYcyxh4IAgazYhhDvZNIS0Qb2qNqBC9q/CQXy0SGGcR/mONTNu9O25S0qy0d7qUaHZcDH4FJlk4Qh/iWef7PoWlWYn3ReEeoDmVYDqaZjZhmvpyPkR5FV50+xQHiLMYhk5+Zs2a2eIkLeRYtaQeysQBxyTU8jovWXidWaYuXicFNiD8x7so4R/Lg3A7gHqqr6gH9yRsVbcQjCoJLz0ysPIs0NiL9+RS2RoKgAA1jLxt+usn58GVYK16MuTsBUnBZp2j7HkQ9y7DWUpL8nvyyDbLfZNW4KfU/yIJmZqbGRBsY4msUkdJ62hFAwjjdBnp3cJJRYBietufP+80Wim2Y0BvcPCFiH6GTq5tqYGPRF553hdM0e3fiVhXTMaXi0eDygtb9Wo2wugGwsIkDkEwU0rlSnOcbRjkKL4k6VGKFr+sbE45yq3cysy+cZcCsc7yjJypQfxz8= smithi072.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNgA5sXxsUsFFCbAae2j4eDx5EVOm3LwRR2LesAxbpVDE8bRYLN3YxpE0uDkyvGlej2AFjtFlTrzIcJ34mmzfkD3CdftcT7a3yssXhLk4snX2fLDKlAjaIrb0pcEzvFMuaSWtklGKSLQ+W7QROfgeoWjHLj/mAbcFe/IvJxfr4loPumu+PgOofGkpl/XiFUe7AOee2J/V3FViejng35ECtEJHeNcIScigh2aM4tyzuqoTmoDNv35e6qjLdqT89F1jbXQiGdVq9zEnXAaY1a2DkyBV4z7LoiCQ12JnLuqG3Q6kEQpOWfXlRErlPdAGm4K375Kq49BVbTqJR7mmAcM23TYQ+8KtPX5GMtxWtVwvBlsdP8P95dXKyDZfU7Q25VXoSRUoh2PAwBzynsNxB0WDofS2QcRcYxk5CwqlXNT2IFhhoZUDsLwtDNUV37+Fs2wgbHcLpJGouGUa1t70WhPe/TgN65rHF9dI/kAerwvxyjk98uf2KS7otqpiYlr7Pg1M= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 3 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 0 - sudo ceph osd pool set cache min_write_recency_for_promote 0 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T21:55:57.309 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T21:55:57.310 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T21:55:57.310 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T21:55:57.312 INFO:teuthology.task.internal:Checking packages... 2023-01-24T21:55:57.340 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T21:55:57.340 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T21:55:57.341 INFO:teuthology.packaging:ref: None 2023-01-24T21:55:57.341 INFO:teuthology.packaging:tag: None 2023-01-24T21:55:57.341 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T21:55:57.342 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T21:55:57.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T21:55:57.510 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T21:55:57.510 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T21:55:57.517 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T21:55:57.518 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T21:55:57.532 INFO:teuthology.task.internal:Saving configuration 2023-01-24T21:55:57.563 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T21:55:57.580 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T21:55:57.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi035.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 21:50:21.605748', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIVrp1jMNiyJ3kCZmF/YrpV8J3Wvv3+cCDO24puesi0DIoGFOzVZVAFfCt0z2IrE9BVO8daw8E+SOPIUhlA/sFg='} 2023-01-24T21:55:57.629 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 21:50:21.604231', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFdmPIvzt0KkkqdEvyAno/FIrd+QDUOlGbYfYf3U2mvBcE9elbzlwo5C3jfbao85e7dfnyojNV+j4wR2OoDJV0M='} 2023-01-24T21:55:57.629 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T21:55:57.639 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T21:55:57.640 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T21:55:57.641 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T21:55:57.797 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f36716e95e0>, signals=[15]) 2023-01-24T21:55:57.798 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T21:55:57.808 INFO:teuthology.task.internal:Opening connections... 2023-01-24T21:55:57.809 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T21:55:57.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T21:55:57.881 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T21:55:57.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T21:55:57.951 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T21:55:57.961 DEBUG:teuthology.orchestra.run.smithi035:> uname -m 2023-01-24T21:55:57.979 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2023-01-24T21:55:57.980 DEBUG:teuthology.orchestra.run.smithi035:> cat /etc/os-release 2023-01-24T21:55:58.038 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T21:55:58.038 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T21:55:58.038 INFO:teuthology.orchestra.run.smithi035.stdout:ID="rhel" 2023-01-24T21:55:58.039 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="fedora" 2023-01-24T21:55:58.039 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8.4" 2023-01-24T21:55:58.039 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8" 2023-01-24T21:55:58.039 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T21:55:58.040 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31" 2023-01-24T21:55:58.040 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T21:55:58.040 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T21:55:58.040 INFO:teuthology.orchestra.run.smithi035.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T21:55:58.041 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T21:55:58.041 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T21:55:58.041 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T21:55:58.041 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T21:55:58.041 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T21:55:58.042 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T21:55:58.043 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2023-01-24T21:55:58.064 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-01-24T21:55:58.082 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-01-24T21:55:58.083 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-01-24T21:55:58.135 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T21:55:58.135 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T21:55:58.136 INFO:teuthology.orchestra.run.smithi072.stdout:ID="rhel" 2023-01-24T21:55:58.136 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="fedora" 2023-01-24T21:55:58.136 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8.4" 2023-01-24T21:55:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2023-01-24T21:55:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T21:55:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2023-01-24T21:55:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T21:55:58.137 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T21:55:58.138 INFO:teuthology.orchestra.run.smithi072.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T21:55:58.138 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T21:55:58.138 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T21:55:58.138 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T21:55:58.138 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T21:55:58.139 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T21:55:58.139 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T21:55:58.139 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-01-24T21:55:58.156 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T21:55:58.173 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T21:55:58.181 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T21:55:58.182 DEBUG:teuthology.orchestra.run.smithi035:> test '!' -e /home/ubuntu/cephtest 2023-01-24T21:55:58.185 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-01-24T21:55:58.200 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T21:55:58.214 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T21:55:58.214 DEBUG:teuthology.orchestra.run.smithi035:> test -z $(ls -A /var/lib/ceph) 2023-01-24T21:55:58.246 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-01-24T21:55:58.261 INFO:teuthology.orchestra.run.smithi072.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T21:55:58.263 INFO:teuthology.orchestra.run.smithi035.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T21:55:58.264 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T21:55:58.335 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T21:55:58.356 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T21:55:58.357 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T21:55:58.358 DEBUG:teuthology.orchestra.run.smithi035:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T21:55:58.359 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T21:55:58.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T21:55:58.375 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-01-24T21:55:58.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T21:55:58.377 DEBUG:teuthology.orchestra.run.smithi035:> uname -r 2023-01-24T21:55:58.429 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:55:58.429 INFO:teuthology.task.kernel:Running kernel on smithi072: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:55:58.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2023-01-24T21:55:58.434 INFO:teuthology.orchestra.run.smithi035.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:55:58.435 INFO:teuthology.task.kernel:Running kernel on smithi035: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:55:58.435 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum install -y kernel 2023-01-24T21:56:00.023 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:00.023 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2023-01-24T21:56:00.815 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:00.815 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-01-24T21:56:01.418 INFO:teuthology.orchestra.run.smithi035.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-01-24T21:56:02.586 INFO:teuthology.orchestra.run.smithi035.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2023-01-24T21:56:03.883 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 38 kB/s | 3.5 kB 00:00 2023-01-24T21:56:05.510 INFO:teuthology.orchestra.run.smithi035.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-01-24T21:56:06.213 INFO:teuthology.orchestra.run.smithi035.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T21:56:06.213 INFO:teuthology.orchestra.run.smithi035.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T21:56:06.214 INFO:teuthology.orchestra.run.smithi035.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T21:56:06.350 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T21:56:06.351 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T21:56:06.352 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T21:56:06.397 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 13 MB 00:02 2023-01-24T21:56:06.448 DEBUG:teuthology.orchestra.run.smithi035:> echo no | sudo yum reinstall kernel || true 2023-01-24T21:56:06.815 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:06.815 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2023-01-24T21:56:07.110 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 09:56:05 PM UTC. 2023-01-24T21:56:07.250 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T21:56:07.251 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:07.252 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:07.273 INFO:teuthology.orchestra.run.smithi035.stderr:Error: No packages marked for reinstall. 2023-01-24T21:56:07.324 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum reinstall -y kernel || true 2023-01-24T21:56:07.689 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:07.689 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2023-01-24T21:56:07.985 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 09:56:05 PM UTC. 2023-01-24T21:56:08.132 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T21:56:08.133 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:08.133 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:08.145 INFO:teuthology.orchestra.run.smithi035.stderr:Error: No packages marked for reinstall. 2023-01-24T21:56:08.202 DEBUG:teuthology.orchestra.run.smithi035:> rpm -q kernel --last | head -n 1 2023-01-24T21:56:08.332 INFO:teuthology.orchestra.run.smithi035.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T21:56:08.334 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:56:08.334 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T21:56:08.334 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T21:56:08.335 DEBUG:teuthology.orchestra.run.smithi035:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T21:56:08.375 INFO:teuthology.orchestra.run.smithi035.stdout:ttyS1 2023-01-24T21:56:08.388 DEBUG:teuthology.parallel:result is None 2023-01-24T21:56:09.195 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-01-24T21:56:09.893 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T21:56:09.894 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T21:56:09.894 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T21:56:10.036 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T21:56:10.037 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T21:56:10.037 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T21:56:10.152 DEBUG:teuthology.orchestra.run.smithi072:> echo no | sudo yum reinstall kernel || true 2023-01-24T21:56:10.495 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:10.496 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-01-24T21:56:10.777 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:01 ago on Tue 24 Jan 2023 09:56:09 PM UTC. 2023-01-24T21:56:10.911 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T21:56:10.911 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:10.912 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:Error: No packages marked for reinstall. 2023-01-24T21:56:11.002 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum reinstall -y kernel || true 2023-01-24T21:56:11.346 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T21:56:11.346 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-01-24T21:56:11.627 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 09:56:09 PM UTC. 2023-01-24T21:56:11.770 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T21:56:11.770 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:11.771 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T21:56:11.795 INFO:teuthology.orchestra.run.smithi072.stderr:Error: No packages marked for reinstall. 2023-01-24T21:56:11.844 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel --last | head -n 1 2023-01-24T21:56:11.970 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T21:56:11.970 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T21:56:11.971 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T21:56:11.971 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T21:56:11.971 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T21:56:11.999 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2023-01-24T21:56:12.018 DEBUG:teuthology.parallel:result is None 2023-01-24T21:56:12.018 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T21:56:12.038 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T21:56:12.039 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T21:56:12.043 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T21:56:12.077 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T21:56:12.087 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T21:56:12.099 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T21:56:12.100 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T21:56:12.106 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T21:56:12.141 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T21:56:12.150 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T21:56:12.151 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T21:56:12.177 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T21:56:12.213 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T21:56:12.217 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T21:56:12.223 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T21:56:12.229 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T21:56:12.231 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T21:56:12.241 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T21:56:12.242 DEBUG:teuthology.orchestra.run.smithi035:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T21:56:12.273 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T21:56:12.299 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T21:56:12.316 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T21:56:12.316 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T21:56:12.346 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T21:56:12.360 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T21:56:12.442 DEBUG:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T21:56:12.484 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T21:56:12.565 DEBUG:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T21:56:12.608 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T21:56:12.609 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T21:56:12.677 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T21:56:12.725 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T21:56:12.760 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T21:56:12.842 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T21:56:12.882 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T21:56:12.883 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T21:56:12.945 DEBUG:teuthology.orchestra.run.smithi035:> sudo service rsyslog restart 2023-01-24T21:56:12.948 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-01-24T21:56:13.014 INFO:teuthology.orchestra.run.smithi035.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T21:56:13.031 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T21:56:13.276 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T21:56:13.285 INFO:teuthology.task.internal:Starting timer... 2023-01-24T21:56:13.285 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T21:56:13.301 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T21:56:13.317 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T21:56:13.318 DEBUG:teuthology.orchestra.run.smithi035:> sudo service auditd rotate 2023-01-24T21:56:13.387 INFO:teuthology.orchestra.run.smithi035.stdout:Rotating logs: [ OK ] 2023-01-24T21:56:13.389 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2023-01-24T21:56:13.451 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: [ OK ] 2023-01-24T21:56:13.453 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T21:56:13.454 DEBUG:teuthology.orchestra.run.smithi035:> /usr/sbin/getenforce 2023-01-24T21:56:13.492 INFO:teuthology.orchestra.run.smithi035.stdout:Permissive 2023-01-24T21:56:13.492 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2023-01-24T21:56:13.514 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2023-01-24T21:56:13.514 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi035.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi072.front.sepia.ceph.com': 'permissive'} 2023-01-24T21:56:13.515 DEBUG:teuthology.orchestra.run.smithi035:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T21:56:13.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T21:56:13.558 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T21:56:13.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T21:56:13.584 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T21:56:13.585 DEBUG:teuthology.orchestra.run.smithi035:> sudo /usr/sbin/setenforce permissive 2023-01-24T21:56:13.625 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2023-01-24T21:56:13.648 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T21:56:13.660 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-24T21:56:13.661 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T21:56:13.693 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T21:56:13.695 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi035.front.sepia.ceph.com,smithi072.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T22:03:05.431 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi035.front.sepia.ceph.com'), Remote(name='ubuntu@smithi072.front.sepia.ceph.com')] 2023-01-24T22:03:05.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2023-01-24T22:03:05.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T22:03:05.503 DEBUG:teuthology.orchestra.run.smithi035:> true 2023-01-24T22:03:05.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-01-24T22:03:05.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T22:03:05.636 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-01-24T22:03:05.656 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T22:03:05.668 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T22:03:05.668 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T22:03:05.669 DEBUG:teuthology.orchestra.run.smithi035:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T22:03:05.671 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T22:03:05.672 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T22:03:05.703 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T22:03:05.719 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T22:03:05.724 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T22:03:05.737 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T22:03:05.753 INFO:teuthology.orchestra.run.smithi035.stderr:sudo: ntpd: command not found 2023-01-24T22:03:05.765 INFO:teuthology.orchestra.run.smithi035.stdout:506 Cannot talk to daemon 2023-01-24T22:03:05.771 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2023-01-24T22:03:05.780 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T22:03:05.781 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2023-01-24T22:03:05.794 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T22:03:05.795 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T22:03:05.807 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T22:03:05.867 INFO:teuthology.orchestra.run.smithi035.stderr:bash: ntpq: command not found 2023-01-24T22:03:05.871 INFO:teuthology.orchestra.run.smithi035.stdout:210 Number of sources = 4 2023-01-24T22:03:05.871 INFO:teuthology.orchestra.run.smithi035.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T22:03:05.871 INFO:teuthology.orchestra.run.smithi035.stdout:=============================================================================== 2023-01-24T22:03:05.872 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.872 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.872 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.873 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.874 INFO:teuthology.orchestra.run.smithi072.stderr:bash: ntpq: command not found 2023-01-24T22:03:05.877 INFO:teuthology.orchestra.run.smithi072.stdout:210 Number of sources = 4 2023-01-24T22:03:05.877 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T22:03:05.877 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-01-24T22:03:05.877 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.877 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.878 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.878 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T22:03:05.878 INFO:teuthology.run_tasks:Running task install... 2023-01-24T22:03:05.887 DEBUG:teuthology.task.install:project ceph 2023-01-24T22:03:05.887 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T22:03:05.888 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T22:03:05.888 INFO:teuthology.task.install:Using flavor: default 2023-01-24T22:03:05.894 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T22:03:05.894 INFO:teuthology.task.install:extra packages: [] 2023-01-24T22:03:05.895 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2023-01-24T22:03:05.950 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2023-01-24T22:03:05.951 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:03:05.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo lsb_release -is 2023-01-24T22:03:05.953 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T22:03:05.954 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2023-01-24T22:03:05.991 INFO:teuthology.orchestra.run.smithi072.stdout:RedHatEnterprise 2023-01-24T22:03:05.992 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:03:05.993 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T22:03:05.993 DEBUG:teuthology.orchestra.run.smithi072:> sudo lsb_release -is 2023-01-24T22:03:06.031 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2023-01-24T22:03:06.033 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:03:06.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T22:03:06.072 INFO:teuthology.orchestra.run.smithi072.stdout:RedHatEnterprise 2023-01-24T22:03:06.073 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:03:06.073 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T22:03:06.193 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T22:03:06.194 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T22:03:06.241 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T22:03:06.241 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T22:03:06.521 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T22:03:06.521 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:03:06.522 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T22:03:06.548 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T22:03:06.548 DEBUG:teuthology.orchestra.run.smithi035:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T22:03:06.579 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T22:03:06.579 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:03:06.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T22:03:06.605 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T22:03:06.605 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T22:03:06.614 DEBUG:teuthology.orchestra.run.smithi035:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T22:03:06.669 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T22:03:06.692 DEBUG:teuthology.orchestra.run.smithi035:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T22:03:06.745 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T22:03:06.759 INFO:teuthology.orchestra.run.smithi035.stdout:check_obsoletes = 1 2023-01-24T22:03:06.760 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum clean all 2023-01-24T22:03:06.808 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2023-01-24T22:03:06.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2023-01-24T22:03:07.170 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:03:07.192 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:03:07.516 INFO:teuthology.orchestra.run.smithi035.stdout:44 files removed 2023-01-24T22:03:07.530 INFO:teuthology.orchestra.run.smithi072.stdout:44 files removed 2023-01-24T22:03:07.567 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-radosgw 2023-01-24T22:03:07.579 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw 2023-01-24T22:03:07.920 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:03:07.934 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:03:09.025 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 117 kB/s | 78 kB 00:00 2023-01-24T22:03:09.087 INFO:teuthology.orchestra.run.smithi035.stdout:ceph packages for x86_64 112 kB/s | 78 kB 00:00 2023-01-24T22:03:09.664 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 15 kB/s | 9.2 kB 00:00 2023-01-24T22:03:09.744 INFO:teuthology.orchestra.run.smithi035.stdout:ceph noarch packages 15 kB/s | 9.2 kB 00:00 2023-01-24T22:03:10.159 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2023-01-24T22:03:10.183 INFO:teuthology.orchestra.run.smithi035.stdout:ceph source packages 3.6 kB/s | 1.5 kB 00:00 2023-01-24T22:03:10.271 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-01-24T22:03:10.290 INFO:teuthology.orchestra.run.smithi035.stdout:Copr repo for python3-asyncssh owned by ceph 47 kB/s | 3.5 kB 00:00 2023-01-24T22:03:11.215 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-24T22:03:11.229 INFO:teuthology.orchestra.run.smithi035.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-24T22:03:13.994 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-01-24T22:03:14.178 INFO:teuthology.orchestra.run.smithi035.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-01-24T22:03:14.336 INFO:teuthology.orchestra.run.smithi072.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 124 MB/s | 33 MB 00:00 2023-01-24T22:03:14.568 INFO:teuthology.orchestra.run.smithi035.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 106 MB/s | 33 MB 00:00 2023-01-24T22:03:19.581 INFO:teuthology.orchestra.run.smithi072.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 108 MB/s | 38 MB 00:00 2023-01-24T22:03:20.010 INFO:teuthology.orchestra.run.smithi035.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 109 MB/s | 38 MB 00:00 2023-01-24T22:03:28.245 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:03:28.248 INFO:teuthology.orchestra.run.smithi072.stdout:=================================================================================================== 2023-01-24T22:03:28.248 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:03:28.248 INFO:teuthology.orchestra.run.smithi072.stdout:=================================================================================================== 2023-01-24T22:03:28.249 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:03:28.249 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T22:03:28.249 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-01-24T22:03:28.249 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T22:03:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T22:03:28.250 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:03:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T22:03:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T22:03:28.251 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T22:03:28.251 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T22:03:28.251 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T22:03:28.251 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T22:03:28.251 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T22:03:28.252 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T22:03:28.252 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T22:03:28.252 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T22:03:28.252 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T22:03:28.252 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T22:03:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T22:03:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T22:03:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T22:03:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T22:03:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:03:28.254 INFO:teuthology.orchestra.run.smithi072.stdout:=================================================================================================== 2023-01-24T22:03:28.255 INFO:teuthology.orchestra.run.smithi072.stdout:Install 21 Packages 2023-01-24T22:03:28.255 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 2 Packages 2023-01-24T22:03:28.255 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:03:28.256 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 63 M 2023-01-24T22:03:28.256 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:03:29.176 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:03:29.179 INFO:teuthology.orchestra.run.smithi035.stdout:=================================================================================================== 2023-01-24T22:03:29.179 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:03:29.179 INFO:teuthology.orchestra.run.smithi035.stdout:=================================================================================================== 2023-01-24T22:03:29.180 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:03:29.180 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T22:03:29.180 INFO:teuthology.orchestra.run.smithi035.stdout:Upgrading: 2023-01-24T22:03:29.180 INFO:teuthology.orchestra.run.smithi035.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T22:03:29.180 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T22:03:29.181 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:03:29.181 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T22:03:29.181 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T22:03:29.181 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T22:03:29.181 INFO:teuthology.orchestra.run.smithi035.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T22:03:29.182 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T22:03:29.182 INFO:teuthology.orchestra.run.smithi035.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T22:03:29.182 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T22:03:29.182 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T22:03:29.182 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T22:03:29.183 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T22:03:29.183 INFO:teuthology.orchestra.run.smithi035.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T22:03:29.183 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T22:03:29.183 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T22:03:29.183 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T22:03:29.184 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T22:03:29.184 INFO:teuthology.orchestra.run.smithi035.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T22:03:29.184 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T22:03:29.184 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T22:03:29.184 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout:=================================================================================================== 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout:Install 21 Packages 2023-01-24T22:03:29.185 INFO:teuthology.orchestra.run.smithi035.stdout:Upgrade 2 Packages 2023-01-24T22:03:29.186 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:03:29.186 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 63 M 2023-01-24T22:03:29.187 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:03:29.690 INFO:teuthology.orchestra.run.smithi072.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 3.6 MB/s | 5.1 MB 00:01 2023-01-24T22:03:29.760 INFO:teuthology.orchestra.run.smithi072.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 350 kB/s | 24 kB 00:00 2023-01-24T22:03:29.918 INFO:teuthology.orchestra.run.smithi072.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 4.9 MB/s | 793 kB 00:00 2023-01-24T22:03:30.386 INFO:teuthology.orchestra.run.smithi072.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 1.1 MB/s | 506 kB 00:00 2023-01-24T22:03:31.993 INFO:teuthology.orchestra.run.smithi072.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 3.4 MB/s | 5.5 MB 00:01 2023-01-24T22:03:32.057 INFO:teuthology.orchestra.run.smithi072.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 700 kB/s | 45 kB 00:00 2023-01-24T22:03:32.128 INFO:teuthology.orchestra.run.smithi072.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.5 MB/s | 108 kB 00:00 2023-01-24T22:03:32.267 INFO:teuthology.orchestra.run.smithi072.stdout:(8/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8.x 4.1 MB/s | 17 MB 00:04 2023-01-24T22:03:32.269 INFO:teuthology.orchestra.run.smithi072.stdout:(9/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 1.3 MB/s | 190 kB 00:00 2023-01-24T22:03:32.390 INFO:teuthology.orchestra.run.smithi035.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 1.6 MB/s | 5.1 MB 00:03 2023-01-24T22:03:32.416 INFO:teuthology.orchestra.run.smithi072.stdout:(10/23): python3-rados-17.2.5-727.g49f8fb05.el8 2.4 MB/s | 370 kB 00:00 2023-01-24T22:03:32.424 INFO:teuthology.orchestra.run.smithi072.stdout:(11/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 2.2 MB/s | 357 kB 00:00 2023-01-24T22:03:32.451 INFO:teuthology.orchestra.run.smithi035.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 400 kB/s | 24 kB 00:00 2023-01-24T22:03:32.469 INFO:teuthology.orchestra.run.smithi072.stdout:(12/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 2.0 MB/s | 107 kB 00:00 2023-01-24T22:03:32.652 INFO:teuthology.orchestra.run.smithi035.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 3.9 MB/s | 793 kB 00:00 2023-01-24T22:03:32.666 INFO:teuthology.orchestra.run.smithi072.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 299 kB/s | 59 kB 00:00 2023-01-24T22:03:32.757 INFO:teuthology.orchestra.run.smithi035.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 4.7 MB/s | 506 kB 00:00 2023-01-24T22:03:32.965 INFO:teuthology.orchestra.run.smithi072.stdout:(14/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 4.9 MB/s | 23 MB 00:04 2023-01-24T22:03:32.966 INFO:teuthology.orchestra.run.smithi072.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 250 kB/s | 75 kB 00:00 2023-01-24T22:03:32.967 INFO:teuthology.orchestra.run.smithi072.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 564 kB/s | 306 kB 00:00 2023-01-24T22:03:33.000 INFO:teuthology.orchestra.run.smithi072.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 1.3 MB/s | 44 kB 00:00 2023-01-24T22:03:33.006 INFO:teuthology.orchestra.run.smithi072.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.5 MB/s | 259 kB 00:00 2023-01-24T22:03:33.021 INFO:teuthology.orchestra.run.smithi072.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 15 MB/s | 353 kB 00:00 2023-01-24T22:03:33.028 INFO:teuthology.orchestra.run.smithi072.stdout:(20/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.2 MB/s | 47 kB 00:00 2023-01-24T22:03:33.680 INFO:teuthology.orchestra.run.smithi072.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2023-01-24T22:03:33.828 INFO:teuthology.orchestra.run.smithi035.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 5.1 MB/s | 5.5 MB 00:01 2023-01-24T22:03:33.881 INFO:teuthology.orchestra.run.smithi035.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 857 kB/s | 45 kB 00:00 2023-01-24T22:03:33.941 INFO:teuthology.orchestra.run.smithi035.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.8 MB/s | 108 kB 00:00 2023-01-24T22:03:33.951 INFO:teuthology.orchestra.run.smithi072.stdout:(22/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 3.9 MB/s | 3.6 MB 00:00 2023-01-24T22:03:34.013 INFO:teuthology.orchestra.run.smithi035.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 2.6 MB/s | 190 kB 00:00 2023-01-24T22:03:34.070 INFO:teuthology.orchestra.run.smithi072.stdout:(23/23): librados2-17.2.5-727.g49f8fb05.el8.x86 3.3 MB/s | 3.5 MB 00:01 2023-01-24T22:03:34.074 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:03:34.075 INFO:teuthology.orchestra.run.smithi072.stdout:Total 11 MB/s | 63 MB 00:05 2023-01-24T22:03:34.076 INFO:teuthology.orchestra.run.smithi035.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 5.8 MB/s | 370 kB 00:00 2023-01-24T22:03:34.079 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:03:34.126 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:03:34.126 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:03:34.156 INFO:teuthology.orchestra.run.smithi035.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 4.4 MB/s | 357 kB 00:00 2023-01-24T22:03:34.193 INFO:teuthology.orchestra.run.smithi035.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 2.8 MB/s | 107 kB 00:00 2023-01-24T22:03:34.552 INFO:teuthology.orchestra.run.smithi035.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 853 kB/s | 306 kB 00:00 2023-01-24T22:03:34.585 INFO:teuthology.orchestra.run.smithi035.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 1.8 MB/s | 59 kB 00:00 2023-01-24T22:03:34.642 INFO:teuthology.orchestra.run.smithi035.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-01-24T22:03:34.957 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:03:34.961 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:03:35.256 INFO:teuthology.orchestra.run.smithi035.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 2.8 MB/s | 1.7 MB 00:00 2023-01-24T22:03:35.286 INFO:teuthology.orchestra.run.smithi035.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 1.4 MB/s | 44 kB 00:00 2023-01-24T22:03:35.302 INFO:teuthology.orchestra.run.smithi035.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-01-24T22:03:35.319 INFO:teuthology.orchestra.run.smithi035.stdout:(18/23): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-01-24T22:03:35.332 INFO:teuthology.orchestra.run.smithi035.stdout:(19/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.6 MB/s | 47 kB 00:00 2023-01-24T22:03:35.890 INFO:teuthology.orchestra.run.smithi035.stdout:(20/23): librados2-17.2.5-727.g49f8fb05.el8.x86 6.2 MB/s | 3.5 MB 00:00 2023-01-24T22:03:36.104 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:03:36.314 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T22:03:36.403 INFO:teuthology.orchestra.run.smithi035.stdout:(21/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 7.0 MB/s | 3.6 MB 00:00 2023-01-24T22:03:36.702 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T22:03:36.923 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:03:37.009 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:03:37.244 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T22:03:37.523 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T22:03:37.731 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:03:38.132 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:03:38.348 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:03:38.422 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:03:38.622 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-01-24T22:03:38.831 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T22:03:39.053 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T22:03:39.789 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T22:03:39.947 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:03:40.016 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:03:40.133 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T22:03:40.351 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T22:03:40.436 INFO:teuthology.orchestra.run.smithi035.stdout:(22/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 1.5 MB/s | 17 MB 00:11 2023-01-24T22:03:40.599 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T22:03:40.834 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T22:03:40.972 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T22:03:41.041 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T22:03:41.282 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T22:03:41.318 INFO:teuthology.orchestra.run.smithi035.stdout:(23/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 1.9 MB/s | 23 MB 00:12 2023-01-24T22:03:41.322 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:03:41.323 INFO:teuthology.orchestra.run.smithi035.stdout:Total 5.2 MB/s | 63 MB 00:12 2023-01-24T22:03:41.328 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:03:41.381 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:03:41.381 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:03:41.506 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T22:03:41.778 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T22:03:41.973 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T22:03:42.252 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:03:42.257 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:03:43.523 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:03:43.763 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T22:03:44.173 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T22:03:44.421 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:03:44.539 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:03:44.723 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T22:03:44.992 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T22:03:45.171 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:03:45.598 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:03:45.746 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:03:45.843 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:03:46.050 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-01-24T22:03:46.262 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T22:03:46.512 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T22:03:46.788 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:47.031 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:47.218 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T22:03:47.610 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:47.637 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:03:47.743 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:03:47.867 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T22:03:47.919 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T22:03:48.012 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T22:03:48.086 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T22:03:48.138 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:03:48.335 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T22:03:48.532 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T22:03:48.662 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T22:03:48.739 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T22:03:48.932 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T22:03:49.162 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T22:03:49.438 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T22:03:49.638 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T22:03:54.719 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:54.962 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:55.583 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T22:03:55.929 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T22:03:56.012 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T22:03:56.111 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:04:07.653 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:04:07.653 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2023-01-24T22:04:07.653 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2023-01-24T22:04:07.653 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-01-24T22:04:07.654 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /run 2023-01-24T22:04:07.654 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2023-01-24T22:04:07.654 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2023-01-24T22:04:07.654 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2023-01-24T22:04:07.654 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2023-01-24T22:04:07.655 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-01-24T22:04:07.655 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:09.424 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T22:04:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T22:04:09.504 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T22:04:09.712 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T22:04:09.713 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:10.213 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:10.213 INFO:teuthology.orchestra.run.smithi072.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T22:04:10.213 INFO:teuthology.orchestra.run.smithi072.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T22:04:10.214 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T22:04:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:04:10.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T22:04:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T22:04:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:04:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:04:10.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T22:04:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T22:04:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T22:04:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:04:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T22:04:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T22:04:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T22:04:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T22:04:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T22:04:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T22:04:10.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T22:04:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T22:04:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-24T22:04:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-01-24T22:04:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-01-24T22:04:10.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:04:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T22:04:10.480 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T22:04:12.031 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:12.031 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:53 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:13.923 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:04:14.032 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:14.032 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-01-24T22:04:14.032 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.033 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.033 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:04:14.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T22:04:14.034 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.035 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T22:04:14.035 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.035 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T22:04:14.035 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T22:04:14.035 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.036 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.036 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.036 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T22:04:14.036 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.037 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.037 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:14.037 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T22:04:14.037 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:14.037 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:04:14.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-test 2023-01-24T22:04:14.487 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:04:14.487 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /sys 2023-01-24T22:04:14.487 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /proc 2023-01-24T22:04:14.487 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /dev 2023-01-24T22:04:14.488 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /run 2023-01-24T22:04:14.488 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /mnt 2023-01-24T22:04:14.488 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /var/tmp 2023-01-24T22:04:14.488 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /home 2023-01-24T22:04:14.488 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /tmp 2023-01-24T22:04:14.489 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /dev 2023-01-24T22:04:14.491 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:14.527 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:15.408 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:56 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:16.318 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T22:04:16.393 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T22:04:16.394 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T22:04:16.586 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T22:04:16.586 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:17.102 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:17.103 INFO:teuthology.orchestra.run.smithi035.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T22:04:17.103 INFO:teuthology.orchestra.run.smithi035.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T22:04:17.103 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:17.235 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:04:17.236 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================== 2023-01-24T22:04:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:04:17.236 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================== 2023-01-24T22:04:17.236 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:04:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T22:04:17.237 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:04:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T22:04:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T22:04:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T22:04:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T22:04:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T22:04:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:17.238 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:04:17.238 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================== 2023-01-24T22:04:17.239 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2023-01-24T22:04:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:17.239 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 53 M 2023-01-24T22:04:17.240 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 233 M 2023-01-24T22:04:17.240 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:04:17.241 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 20 MB/s | 70 kB 00:00 2023-01-24T22:04:17.276 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 4.7 MB/s | 161 kB 00:00 2023-01-24T22:04:17.291 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-01-24T22:04:17.307 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-01-24T22:04:17.407 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 1.0 MB/s | 174 kB 00:00 2023-01-24T22:04:17.413 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T22:04:17.413 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T22:04:17.413 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T22:04:17.414 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T22:04:17.414 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T22:04:17.414 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T22:04:17.414 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T22:04:17.414 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T22:04:17.415 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T22:04:17.415 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T22:04:17.415 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T22:04:17.415 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T22:04:17.415 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T22:04:17.416 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T22:04:17.416 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T22:04:17.416 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T22:04:17.416 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T22:04:17.417 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T22:04:17.417 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-24T22:04:17.417 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-01-24T22:04:17.417 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-01-24T22:04:17.418 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T22:04:17.418 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T22:04:17.418 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T22:04:18.889 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T22:04:18.889 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:00:59 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:20.014 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 19 MB/s | 52 MB 00:02 2023-01-24T22:04:20.017 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:04:20.018 INFO:teuthology.orchestra.run.smithi072.stdout:Total 19 MB/s | 53 MB 00:02 2023-01-24T22:04:20.021 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:04:20.044 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:04:20.045 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:04:20.397 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:04:20.401 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:04:20.826 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:04:20.914 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:04:20.944 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:20.945 INFO:teuthology.orchestra.run.smithi035.stdout:Upgraded: 2023-01-24T22:04:20.945 INFO:teuthology.orchestra.run.smithi035.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.945 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.945 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:04:20.946 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.946 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.946 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.946 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.947 INFO:teuthology.orchestra.run.smithi035.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T22:04:20.947 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.947 INFO:teuthology.orchestra.run.smithi035.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T22:04:20.947 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T22:04:20.947 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.948 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T22:04:20.948 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.948 INFO:teuthology.orchestra.run.smithi035.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T22:04:20.948 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T22:04:20.948 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.949 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.949 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.949 INFO:teuthology.orchestra.run.smithi035.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T22:04:20.949 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.949 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.950 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:20.950 INFO:teuthology.orchestra.run.smithi035.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T22:04:20.950 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:20.950 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:04:21.088 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-test 2023-01-24T22:04:21.112 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T22:04:21.326 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T22:04:21.405 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T22:04:21.467 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:21.593 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T22:04:21.766 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T22:04:21.942 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T22:04:22.346 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:03 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:24.346 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:04:24.347 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================== 2023-01-24T22:04:24.347 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:04:24.347 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================== 2023-01-24T22:04:24.347 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:04:24.348 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T22:04:24.348 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:04:24.348 INFO:teuthology.orchestra.run.smithi035.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T22:04:24.348 INFO:teuthology.orchestra.run.smithi035.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:04:24.349 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================== 2023-01-24T22:04:24.350 INFO:teuthology.orchestra.run.smithi035.stdout:Install 6 Packages 2023-01-24T22:04:24.350 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:24.350 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 53 M 2023-01-24T22:04:24.351 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 233 M 2023-01-24T22:04:24.351 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:04:24.353 INFO:teuthology.orchestra.run.smithi035.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-01-24T22:04:24.384 INFO:teuthology.orchestra.run.smithi035.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.1 MB/s | 161 kB 00:00 2023-01-24T22:04:24.397 INFO:teuthology.orchestra.run.smithi035.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 14 MB/s | 187 kB 00:00 2023-01-24T22:04:24.412 INFO:teuthology.orchestra.run.smithi035.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 21 MB/s | 302 kB 00:00 2023-01-24T22:04:24.543 INFO:teuthology.orchestra.run.smithi035.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 900 kB/s | 174 kB 00:00 2023-01-24T22:04:27.027 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T22:04:27.257 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T22:04:27.520 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T22:04:27.520 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T22:04:27.520 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T22:04:27.520 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T22:04:27.521 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T22:04:27.521 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T22:04:28.284 INFO:teuthology.orchestra.run.smithi035.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 13 MB/s | 52 MB 00:03 2023-01-24T22:04:28.287 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:04:28.287 INFO:teuthology.orchestra.run.smithi035.stdout:Total 13 MB/s | 53 MB 00:03 2023-01-24T22:04:28.291 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:04:28.318 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:04:28.318 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:04:28.646 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T22:04:28.647 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:09 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:28.711 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:04:28.716 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:04:29.299 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:04:29.522 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T22:04:29.732 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T22:04:29.818 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T22:04:30.012 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T22:04:30.165 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T22:04:30.307 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T22:04:30.512 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:04:30.609 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:30.610 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:04:30.610 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:30.610 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.5-12.el8.x86_64 2023-01-24T22:04:30.610 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:30.611 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T22:04:30.611 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T22:04:30.611 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T22:04:30.611 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:30.611 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:04:30.723 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph 2023-01-24T22:04:31.068 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:31.933 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:12 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:33.713 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:04:33.716 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================== 2023-01-24T22:04:33.716 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:04:33.716 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================== 2023-01-24T22:04:33.716 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:04:33.717 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T22:04:33.717 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:04:33.717 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T22:04:33.717 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T22:04:33.718 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T22:04:33.719 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T22:04:33.719 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T22:04:33.719 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T22:04:33.719 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T22:04:33.720 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T22:04:33.720 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T22:04:33.720 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T22:04:33.720 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T22:04:33.720 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T22:04:33.721 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T22:04:33.722 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T22:04:33.722 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T22:04:33.722 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T22:04:33.722 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T22:04:33.722 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:33.723 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:04:33.723 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================== 2023-01-24T22:04:33.723 INFO:teuthology.orchestra.run.smithi072.stdout:Install 28 Packages 2023-01-24T22:04:33.723 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:33.724 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 27 M 2023-01-24T22:04:33.724 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 105 M 2023-01-24T22:04:33.724 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:04:33.870 INFO:teuthology.orchestra.run.smithi072.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 43 kB/s | 6.4 kB 00:00 2023-01-24T22:04:34.949 INFO:teuthology.orchestra.run.smithi072.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 1.3 MB/s | 1.6 MB 00:01 2023-01-24T22:04:35.412 INFO:teuthology.orchestra.run.smithi072.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 1.4 MB/s | 2.3 MB 00:01 2023-01-24T22:04:35.513 INFO:teuthology.orchestra.run.smithi072.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 2.2 MB/s | 233 kB 00:00 2023-01-24T22:04:35.618 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T22:04:35.625 INFO:teuthology.orchestra.run.smithi072.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 390 kB/s | 44 kB 00:00 2023-01-24T22:04:35.710 INFO:teuthology.orchestra.run.smithi072.stdout:(6/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.1 MB/s | 185 kB 00:00 2023-01-24T22:04:35.740 INFO:teuthology.orchestra.run.smithi072.stdout:(7/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 5.6 MB/s | 173 kB 00:00 2023-01-24T22:04:35.798 INFO:teuthology.orchestra.run.smithi072.stdout:(8/28): python3-cherrypy-18.4.0-1.el8.noarch.rp 6.5 MB/s | 384 kB 00:00 2023-01-24T22:04:35.828 INFO:teuthology.orchestra.run.smithi072.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 382 kB/s | 11 kB 00:00 2023-01-24T22:04:35.857 INFO:teuthology.orchestra.run.smithi072.stdout:(10/28): python3-jaraco-functools-2.0-4.el8.noa 616 kB/s | 18 kB 00:00 2023-01-24T22:04:35.872 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T22:04:35.886 INFO:teuthology.orchestra.run.smithi072.stdout:(11/28): python3-logutils-0.3.5-11.el8.noarch.r 1.7 MB/s | 49 kB 00:00 2023-01-24T22:04:35.915 INFO:teuthology.orchestra.run.smithi072.stdout:(12/28): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2023-01-24T22:04:35.975 INFO:teuthology.orchestra.run.smithi072.stdout:(13/28): python3-pecan-1.3.2-9.el8.noarch.rpm 4.6 MB/s | 283 kB 00:00 2023-01-24T22:04:36.004 INFO:teuthology.orchestra.run.smithi072.stdout:(14/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_ 1.8 MB/s | 3.9 MB 00:02 2023-01-24T22:04:36.005 INFO:teuthology.orchestra.run.smithi072.stdout:(15/28): python3-portend-2.6-1.el8.noarch.rpm 516 kB/s | 16 kB 00:00 2023-01-24T22:04:36.034 INFO:teuthology.orchestra.run.smithi072.stdout:(16/28): python3-singledispatch-3.4.0.3-18.el8. 847 kB/s | 24 kB 00:00 2023-01-24T22:04:36.062 INFO:teuthology.orchestra.run.smithi072.stdout:(17/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T22:04:36.089 INFO:teuthology.orchestra.run.smithi072.stdout:(18/28): python3-simplegeneric-0.8.1-17.el8.noa 222 kB/s | 19 kB 00:00 2023-01-24T22:04:36.091 INFO:teuthology.orchestra.run.smithi072.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 952 kB/s | 27 kB 00:00 2023-01-24T22:04:36.148 INFO:teuthology.orchestra.run.smithi072.stdout:(20/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.4 MB/s | 251 kB 00:00 2023-01-24T22:04:36.167 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T22:04:36.167 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T22:04:36.168 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T22:04:36.168 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T22:04:36.168 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T22:04:36.168 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T22:04:36.178 INFO:teuthology.orchestra.run.smithi072.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.8 MB/s | 85 kB 00:00 2023-01-24T22:04:36.202 INFO:teuthology.orchestra.run.smithi072.stdout:(22/28): python3-waitress-1.2.1-2.el8.1.noarch. 2.1 MB/s | 241 kB 00:00 2023-01-24T22:04:36.207 INFO:teuthology.orchestra.run.smithi072.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 813 kB/s | 23 kB 00:00 2023-01-24T22:04:36.236 INFO:teuthology.orchestra.run.smithi072.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 13 MB/s | 457 kB 00:00 2023-01-24T22:04:36.237 INFO:teuthology.orchestra.run.smithi072.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.3 MB/s | 39 kB 00:00 2023-01-24T22:04:36.250 INFO:teuthology.orchestra.run.smithi072.stdout:(26/28): python3-pytz-2017.2-9.el8.noarch.rpm 4.2 MB/s | 54 kB 00:00 2023-01-24T22:04:36.251 INFO:teuthology.orchestra.run.smithi072.stdout:(27/28): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-01-24T22:04:36.587 INFO:teuthology.orchestra.run.smithi072.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 10 MB/s | 17 MB 00:01 2023-01-24T22:04:36.590 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:04:36.591 INFO:teuthology.orchestra.run.smithi072.stdout:Total 9.5 MB/s | 27 MB 00:02 2023-01-24T22:04:36.595 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:04:36.631 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:04:36.631 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:04:37.012 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:04:37.016 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:04:37.263 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T22:04:37.263 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:18 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:38.054 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:04:38.324 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T22:04:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T22:04:38.719 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T22:04:38.981 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T22:04:39.154 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:04:39.261 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:39.261 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:04:39.261 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:39.262 INFO:teuthology.orchestra.run.smithi035.stdout: jq-1.5-12.el8.x86_64 2023-01-24T22:04:39.262 INFO:teuthology.orchestra.run.smithi035.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:39.262 INFO:teuthology.orchestra.run.smithi035.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T22:04:39.262 INFO:teuthology.orchestra.run.smithi035.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T22:04:39.262 INFO:teuthology.orchestra.run.smithi035.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T22:04:39.263 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:39.263 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:04:39.326 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T22:04:39.379 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph 2023-01-24T22:04:39.541 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T22:04:39.747 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:39.757 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T22:04:39.957 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T22:04:40.139 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T22:04:40.353 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T22:04:40.550 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T22:04:40.653 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:21 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:40.735 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T22:04:40.879 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T22:04:41.102 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T22:04:41.352 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T22:04:41.546 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T22:04:41.800 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T22:04:42.037 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T22:04:42.487 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T22:04:42.658 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:04:42.661 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================== 2023-01-24T22:04:42.662 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:04:42.662 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================== 2023-01-24T22:04:42.662 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:04:42.662 INFO:teuthology.orchestra.run.smithi035.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T22:04:42.662 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:04:42.663 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T22:04:42.663 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T22:04:42.663 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T22:04:42.663 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T22:04:42.663 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T22:04:42.664 INFO:teuthology.orchestra.run.smithi035.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T22:04:42.664 INFO:teuthology.orchestra.run.smithi035.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T22:04:42.664 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T22:04:42.664 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T22:04:42.665 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T22:04:42.665 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T22:04:42.665 INFO:teuthology.orchestra.run.smithi035.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T22:04:42.665 INFO:teuthology.orchestra.run.smithi035.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T22:04:42.665 INFO:teuthology.orchestra.run.smithi035.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T22:04:42.666 INFO:teuthology.orchestra.run.smithi035.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T22:04:42.666 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T22:04:42.666 INFO:teuthology.orchestra.run.smithi035.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T22:04:42.666 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T22:04:42.666 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T22:04:42.667 INFO:teuthology.orchestra.run.smithi035.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T22:04:42.667 INFO:teuthology.orchestra.run.smithi035.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T22:04:42.667 INFO:teuthology.orchestra.run.smithi035.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T22:04:42.667 INFO:teuthology.orchestra.run.smithi035.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T22:04:42.667 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T22:04:42.668 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T22:04:42.668 INFO:teuthology.orchestra.run.smithi035.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T22:04:42.670 INFO:teuthology.orchestra.run.smithi035.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T22:04:42.670 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:42.671 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:04:42.671 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================== 2023-01-24T22:04:42.671 INFO:teuthology.orchestra.run.smithi035.stdout:Install 28 Packages 2023-01-24T22:04:42.671 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:42.672 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 27 M 2023-01-24T22:04:42.672 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 105 M 2023-01-24T22:04:42.673 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:04:42.778 INFO:teuthology.orchestra.run.smithi035.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 58 kB/s | 6.4 kB 00:00 2023-01-24T22:04:43.097 INFO:teuthology.orchestra.run.smithi035.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 1.6 MB 00:00 2023-01-24T22:04:43.210 INFO:teuthology.orchestra.run.smithi035.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 4.2 MB/s | 2.3 MB 00:00 2023-01-24T22:04:43.249 INFO:teuthology.orchestra.run.smithi035.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 5.9 MB/s | 233 kB 00:00 2023-01-24T22:04:43.442 INFO:teuthology.orchestra.run.smithi035.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 226 kB/s | 44 kB 00:00 2023-01-24T22:04:43.611 INFO:teuthology.orchestra.run.smithi035.stdout:(6/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.1 MB/s | 185 kB 00:00 2023-01-24T22:04:43.683 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T22:04:43.712 INFO:teuthology.orchestra.run.smithi035.stdout:(7/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2023-01-24T22:04:43.818 INFO:teuthology.orchestra.run.smithi035.stdout:(8/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 3.8 MB/s | 3.9 MB 00:01 2023-01-24T22:04:43.893 INFO:teuthology.orchestra.run.smithi035.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-01-24T22:04:43.916 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T22:04:43.937 INFO:teuthology.orchestra.run.smithi035.stdout:(10/28): python3-cherrypy-18.4.0-1.el8.noarch.r 1.7 MB/s | 384 kB 00:00 2023-01-24T22:04:43.943 INFO:teuthology.orchestra.run.smithi035.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 353 kB/s | 18 kB 00:00 2023-01-24T22:04:43.981 INFO:teuthology.orchestra.run.smithi035.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.1 MB/s | 49 kB 00:00 2023-01-24T22:04:44.020 INFO:teuthology.orchestra.run.smithi035.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 774 kB/s | 59 kB 00:00 2023-01-24T22:04:44.054 INFO:teuthology.orchestra.run.smithi035.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 451 kB/s | 16 kB 00:00 2023-01-24T22:04:44.071 INFO:teuthology.orchestra.run.smithi035.stdout:(15/28): python3-pecan-1.3.2-9.el8.noarch.rpm 3.1 MB/s | 283 kB 00:00 2023-01-24T22:04:44.086 INFO:teuthology.orchestra.run.smithi035.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 591 kB/s | 19 kB 00:00 2023-01-24T22:04:44.101 INFO:teuthology.orchestra.run.smithi035.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 795 kB/s | 24 kB 00:00 2023-01-24T22:04:44.116 INFO:teuthology.orchestra.run.smithi035.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 984 kB/s | 29 kB 00:00 2023-01-24T22:04:44.129 INFO:teuthology.orchestra.run.smithi035.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 992 kB/s | 27 kB 00:00 2023-01-24T22:04:44.200 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T22:04:44.211 INFO:teuthology.orchestra.run.smithi035.stdout:(20/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.0 MB/s | 251 kB 00:00 2023-01-24T22:04:44.240 INFO:teuthology.orchestra.run.smithi035.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 3.0 MB/s | 85 kB 00:00 2023-01-24T22:04:44.257 INFO:teuthology.orchestra.run.smithi035.stdout:(22/28): python3-waitress-1.2.1-2.el8.1.noarch. 1.7 MB/s | 241 kB 00:00 2023-01-24T22:04:44.276 INFO:teuthology.orchestra.run.smithi035.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 635 kB/s | 23 kB 00:00 2023-01-24T22:04:44.299 INFO:teuthology.orchestra.run.smithi035.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 11 MB/s | 457 kB 00:00 2023-01-24T22:04:44.306 INFO:teuthology.orchestra.run.smithi035.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.3 MB/s | 39 kB 00:00 2023-01-24T22:04:44.317 INFO:teuthology.orchestra.run.smithi035.stdout:(26/28): python3-mako-1.0.6-13.el8.noarch.rpm 8.9 MB/s | 157 kB 00:00 2023-01-24T22:04:44.319 INFO:teuthology.orchestra.run.smithi035.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 4.0 MB/s | 54 kB 00:00 2023-01-24T22:04:44.488 INFO:teuthology.orchestra.run.smithi035.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 12 MB/s | 17 MB 00:01 2023-01-24T22:04:44.491 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:04:44.491 INFO:teuthology.orchestra.run.smithi035.stdout:Total 15 MB/s | 27 MB 00:01 2023-01-24T22:04:44.496 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:04:44.535 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:04:44.535 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:04:44.585 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T22:04:44.753 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T22:04:44.933 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:04:44.938 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:04:46.092 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:04:46.237 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T22:04:46.327 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T22:04:46.436 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T22:04:46.555 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T22:04:46.711 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T22:04:46.826 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T22:04:46.992 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T22:04:47.037 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T22:04:47.251 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T22:04:47.352 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T22:04:47.456 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T22:04:47.474 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T22:04:47.580 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T22:04:47.606 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T22:04:47.820 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T22:04:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T22:04:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T22:04:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T22:04:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T22:04:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T22:04:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T22:04:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T22:04:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T22:04:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T22:04:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T22:04:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T22:04:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T22:04:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T22:04:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T22:04:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T22:04:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T22:04:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T22:04:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T22:04:48.030 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T22:04:48.212 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T22:04:48.401 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T22:04:48.586 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T22:04:48.800 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T22:04:49.388 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T22:04:49.599 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T22:04:49.599 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:30 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:49.906 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T22:04:50.120 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T22:04:50.279 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T22:04:50.553 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T22:04:50.792 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T22:04:51.039 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T22:04:51.397 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T22:04:51.566 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:04:51.665 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:51.666 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:04:51.666 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:51.666 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:51.666 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:51.666 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:04:51.667 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:51.667 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:51.667 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T22:04:51.667 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T22:04:51.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T22:04:51.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T22:04:51.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T22:04:51.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T22:04:51.669 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T22:04:51.669 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T22:04:51.669 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T22:04:51.669 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T22:04:51.670 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T22:04:51.670 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T22:04:51.670 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T22:04:51.670 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T22:04:51.670 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T22:04:51.671 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T22:04:51.671 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T22:04:51.671 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T22:04:51.671 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T22:04:51.672 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T22:04:51.672 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T22:04:51.672 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T22:04:51.672 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:51.673 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:04:51.673 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T22:04:51.805 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-base 2023-01-24T22:04:51.977 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T22:04:52.160 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:52.390 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T22:04:52.580 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T22:04:53.035 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:34 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:54.168 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T22:04:54.338 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T22:04:54.754 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T22:04:54.910 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T22:04:55.137 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T22:04:55.304 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T22:04:55.323 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T22:04:55.429 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T22:04:55.844 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T22:04:55.844 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T22:04:55.844 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T22:04:55.845 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T22:04:55.845 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T22:04:55.845 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T22:04:55.845 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T22:04:55.845 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T22:04:55.846 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T22:04:55.846 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T22:04:55.846 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T22:04:55.846 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T22:04:55.846 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T22:04:55.847 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T22:04:55.847 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T22:04:55.847 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T22:04:55.847 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T22:04:55.847 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T22:04:55.848 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T22:04:55.849 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T22:04:55.849 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T22:04:55.849 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T22:04:55.850 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T22:04:55.921 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:04:56.549 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:04:56.550 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:04:56.550 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:04:56.912 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install cephadm 2023-01-24T22:04:57.271 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:04:57.449 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T22:04:57.450 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:38 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:58.160 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:39 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:04:59.457 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:04:59.565 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:59.566 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:04:59.566 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:59.566 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:59.567 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:59.567 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:04:59.567 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:59.567 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:04:59.568 INFO:teuthology.orchestra.run.smithi035.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T22:04:59.568 INFO:teuthology.orchestra.run.smithi035.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T22:04:59.568 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T22:04:59.568 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T22:04:59.569 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T22:04:59.569 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T22:04:59.569 INFO:teuthology.orchestra.run.smithi035.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T22:04:59.569 INFO:teuthology.orchestra.run.smithi035.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T22:04:59.569 INFO:teuthology.orchestra.run.smithi035.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T22:04:59.570 INFO:teuthology.orchestra.run.smithi035.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T22:04:59.570 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T22:04:59.570 INFO:teuthology.orchestra.run.smithi035.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T22:04:59.570 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T22:04:59.571 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T22:04:59.571 INFO:teuthology.orchestra.run.smithi035.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T22:04:59.571 INFO:teuthology.orchestra.run.smithi035.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T22:04:59.571 INFO:teuthology.orchestra.run.smithi035.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T22:04:59.572 INFO:teuthology.orchestra.run.smithi035.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T22:04:59.572 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T22:04:59.572 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T22:04:59.572 INFO:teuthology.orchestra.run.smithi035.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T22:04:59.573 INFO:teuthology.orchestra.run.smithi035.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T22:04:59.573 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:04:59.573 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:04:59.693 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-base 2023-01-24T22:04:59.950 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:04:59.951 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:04:59.951 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:04:59.951 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:04:59.951 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:04:59.951 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T22:04:59.952 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:59.952 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:04:59.952 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:04:59.952 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:04:59.952 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:04:59.953 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 78 k 2023-01-24T22:04:59.953 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 355 k 2023-01-24T22:04:59.953 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:00.078 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:00.094 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 547 kB/s | 78 kB 00:00 2023-01-24T22:05:00.095 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:00.095 INFO:teuthology.orchestra.run.smithi072.stdout:Total 542 kB/s | 78 kB 00:00 2023-01-24T22:05:00.096 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:00.103 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:00.104 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:00.105 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:00.109 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:00.162 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:01.001 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:41 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:02.459 INFO:teuthology.orchestra.run.smithi035.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:05:02.663 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:02.839 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:02.945 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:02.947 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:05:02.947 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:03.038 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:03.054 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install cephadm 2023-01-24T22:05:03.431 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:04.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:04.050 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:45 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:04.352 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:45 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:05.945 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:05:06.039 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:06.040 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:05:06.040 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:06.040 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:06.040 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:06.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T22:05:06.537 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:07.375 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:07.376 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:07.376 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:05:07.376 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:07.377 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:05:07.377 INFO:teuthology.orchestra.run.smithi035.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T22:05:07.377 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:07.377 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:05:07.377 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:07.378 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:05:07.378 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:07.378 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 78 k 2023-01-24T22:05:07.378 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 355 k 2023-01-24T22:05:07.378 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:05:07.404 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:07.536 INFO:teuthology.orchestra.run.smithi035.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 493 kB/s | 78 kB 00:00 2023-01-24T22:05:07.536 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:07.537 INFO:teuthology.orchestra.run.smithi035.stdout:Total 488 kB/s | 78 kB 00:00 2023-01-24T22:05:07.537 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:05:07.545 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:05:07.545 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:05:07.547 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:05:07.552 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:05:07.612 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:05:09.190 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:09.191 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:05:09.191 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-01-24T22:05:09.191 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:05:09.191 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:05:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T22:05:09.192 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:09.192 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:05:09.192 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:05:09.192 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:05:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:09.193 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 162 k 2023-01-24T22:05:09.193 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 490 k 2023-01-24T22:05:09.193 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:09.383 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 845 kB/s | 162 kB 00:00 2023-01-24T22:05:09.383 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:09.384 INFO:teuthology.orchestra.run.smithi072.stdout:Total 838 kB/s | 162 kB 00:00 2023-01-24T22:05:09.384 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:09.396 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:09.397 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:09.450 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:09.454 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:09.566 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:09.728 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:10.302 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:10.575 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:10.791 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:11.130 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:11.849 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:05:11.849 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:12.978 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:05:13.077 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:13.077 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:05:13.077 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:05:13.077 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:13.078 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:13.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr 2023-01-24T22:05:13.645 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:14.159 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:05:14.264 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:14.264 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:05:14.264 INFO:teuthology.orchestra.run.smithi035.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:14.264 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:14.265 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:14.418 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T22:05:14.534 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:14.811 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:15.698 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:01:56 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:15.853 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:05:16.306 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:16.307 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:05:16.307 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:16.411 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T22:05:16.765 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:17.612 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:58 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:17.628 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:17.629 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:17.629 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repo Size 2023-01-24T22:05:17.629 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:17.629 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:05:17.629 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T22:05:17.630 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:17.630 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:05:17.630 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:05:17.630 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:05:17.630 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:17.631 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 162 k 2023-01-24T22:05:17.631 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 490 k 2023-01-24T22:05:17.632 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:05:18.132 INFO:teuthology.orchestra.run.smithi035.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 322 kB/s | 162 kB 00:00 2023-01-24T22:05:18.133 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:18.133 INFO:teuthology.orchestra.run.smithi035.stdout:Total 321 kB/s | 162 kB 00:00 2023-01-24T22:05:18.134 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:05:18.148 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:05:18.149 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:05:18.208 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:05:18.213 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:05:18.344 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:05:18.534 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:18.955 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:20.035 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T22:05:20.035 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:01 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:20.646 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:20.647 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================================== 2023-01-24T22:05:20.647 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:05:20.647 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================================== 2023-01-24T22:05:20.648 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:05:20.648 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T22:05:20.648 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:05:20.649 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T22:05:20.649 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T22:05:20.649 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T22:05:20.650 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T22:05:20.650 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T22:05:20.651 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:20.651 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:05:20.651 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================================== 2023-01-24T22:05:20.652 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2023-01-24T22:05:20.652 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:20.653 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 2.0 M 2023-01-24T22:05:20.654 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 7.8 M 2023-01-24T22:05:20.654 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:20.920 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 85 kB/s | 23 kB 00:00 2023-01-24T22:05:20.927 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 52 kB/s | 14 kB 00:00 2023-01-24T22:05:21.003 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 407 kB/s | 34 kB 00:00 2023-01-24T22:05:21.030 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.5 MB/s | 43 kB 00:00 2023-01-24T22:05:21.064 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.4 MB/s | 196 kB 00:00 2023-01-24T22:05:21.755 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 1.5 MB/s | 1.6 MB 00:01 2023-01-24T22:05:21.757 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:21.757 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2023-01-24T22:05:21.759 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:21.767 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:21.767 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:21.864 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:21.868 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:22.154 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:22.314 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:05:22.358 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T22:05:22.420 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:22.421 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:05:22.421 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:05:22.421 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:22.421 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:22.539 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr 2023-01-24T22:05:22.584 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T22:05:22.788 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T22:05:22.908 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T22:05:22.919 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:23.237 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T22:05:23.431 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T22:05:23.685 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T22:05:23.685 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T22:05:23.685 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T22:05:23.686 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T22:05:23.686 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T22:05:23.686 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T22:05:23.810 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:04 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:24.686 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T22:05:24.687 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:05 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:25.181 INFO:teuthology.orchestra.run.smithi035.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:05:25.655 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:25.656 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:05:25.657 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:25.752 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T22:05:26.133 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:26.683 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:05:26.780 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:26.780 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:05:26.780 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:26.781 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:26.781 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:26.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T22:05:26.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T22:05:26.781 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T22:05:26.782 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:26.782 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:26.902 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T22:05:27.047 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:08 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:27.261 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:28.370 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:09 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:29.048 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:29.049 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================================== 2023-01-24T22:05:29.049 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:05:29.050 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================================== 2023-01-24T22:05:29.050 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:05:29.050 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T22:05:29.050 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:05:29.051 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T22:05:29.051 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T22:05:29.051 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T22:05:29.052 INFO:teuthology.orchestra.run.smithi035.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T22:05:29.052 INFO:teuthology.orchestra.run.smithi035.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T22:05:29.052 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:29.052 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:05:29.053 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================================== 2023-01-24T22:05:29.053 INFO:teuthology.orchestra.run.smithi035.stdout:Install 6 Packages 2023-01-24T22:05:29.053 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:29.054 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 2.0 M 2023-01-24T22:05:29.054 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 7.8 M 2023-01-24T22:05:29.054 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:05:29.173 INFO:teuthology.orchestra.run.smithi035.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 189 kB/s | 23 kB 00:00 2023-01-24T22:05:29.183 INFO:teuthology.orchestra.run.smithi035.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 110 kB/s | 14 kB 00:00 2023-01-24T22:05:29.257 INFO:teuthology.orchestra.run.smithi035.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 402 kB/s | 34 kB 00:00 2023-01-24T22:05:29.290 INFO:teuthology.orchestra.run.smithi035.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-01-24T22:05:29.320 INFO:teuthology.orchestra.run.smithi035.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.4 MB/s | 196 kB 00:00 2023-01-24T22:05:29.327 INFO:teuthology.orchestra.run.smithi035.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 6.0 MB/s | 1.6 MB 00:00 2023-01-24T22:05:29.330 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:29.330 INFO:teuthology.orchestra.run.smithi035.stdout:Total 7.0 MB/s | 2.0 MB 00:00 2023-01-24T22:05:29.332 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:05:29.341 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:05:29.341 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:05:29.475 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:05:29.480 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:05:29.791 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:05:30.037 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T22:05:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================================== 2023-01-24T22:05:30.226 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:05:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================================== 2023-01-24T22:05:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:05:30.227 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T22:05:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:05:30.227 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T22:05:30.227 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T22:05:30.227 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T22:05:30.228 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T22:05:30.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T22:05:30.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T22:05:30.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T22:05:30.229 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:05:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:====================================================================================================================== 2023-01-24T22:05:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:Install 8 Packages 2023-01-24T22:05:30.229 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 36 M 2023-01-24T22:05:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 199 M 2023-01-24T22:05:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:30.262 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T22:05:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 6.0 MB/s | 225 kB 00:00 2023-01-24T22:05:30.331 INFO:teuthology.orchestra.run.smithi072.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 57 MB/s | 3.7 MB 00:00 2023-01-24T22:05:30.407 INFO:teuthology.orchestra.run.smithi072.stdout:(3/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 63 MB/s | 4.7 MB 00:00 2023-01-24T22:05:30.444 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T22:05:30.562 INFO:teuthology.orchestra.run.smithi072.stdout:(4/8): python3-scipy-1.0.0-20.module+el8.1.0+33 44 MB/s | 15 MB 00:00 2023-01-24T22:05:30.578 INFO:teuthology.orchestra.run.smithi072.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-01-24T22:05:30.590 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T22:05:30.599 INFO:teuthology.orchestra.run.smithi072.stdout:(6/8): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-01-24T22:05:30.612 INFO:teuthology.orchestra.run.smithi072.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 22 MB/s | 4.6 MB 00:00 2023-01-24T22:05:30.875 INFO:teuthology.orchestra.run.smithi072.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 11 MB/s | 7.4 MB 00:00 2023-01-24T22:05:30.876 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:30.877 INFO:teuthology.orchestra.run.smithi072.stdout:Total 55 MB/s | 36 MB 00:00 2023-01-24T22:05:30.885 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:30.904 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:30.904 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:30.993 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T22:05:31.300 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T22:05:31.326 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:31.330 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:31.575 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T22:05:31.575 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T22:05:31.576 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T22:05:31.576 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T22:05:31.576 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T22:05:31.576 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T22:05:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:32.092 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T22:05:32.236 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T22:05:32.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T22:05:32.641 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T22:05:32.641 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:13 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:32.933 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T22:05:33.108 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T22:05:33.641 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T22:05:33.817 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T22:05:34.576 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:05:34.606 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T22:05:34.683 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:34.683 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:05:34.683 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:34.684 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:34.684 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:34.684 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T22:05:34.684 INFO:teuthology.orchestra.run.smithi035.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T22:05:34.684 INFO:teuthology.orchestra.run.smithi035.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T22:05:34.685 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:34.685 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:34.807 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T22:05:34.902 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T22:05:35.125 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T22:05:35.189 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:36.078 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:17 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:37.245 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T22:05:38.041 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:38.042 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================================== 2023-01-24T22:05:38.043 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:05:38.043 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================================== 2023-01-24T22:05:38.043 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:05:38.043 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T22:05:38.044 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:05:38.044 INFO:teuthology.orchestra.run.smithi035.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T22:05:38.044 INFO:teuthology.orchestra.run.smithi035.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T22:05:38.044 INFO:teuthology.orchestra.run.smithi035.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T22:05:38.044 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T22:05:38.045 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T22:05:38.045 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T22:05:38.045 INFO:teuthology.orchestra.run.smithi035.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T22:05:38.045 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:38.045 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:05:38.046 INFO:teuthology.orchestra.run.smithi035.stdout:====================================================================================================================== 2023-01-24T22:05:38.046 INFO:teuthology.orchestra.run.smithi035.stdout:Install 8 Packages 2023-01-24T22:05:38.046 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:38.047 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 36 M 2023-01-24T22:05:38.047 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 199 M 2023-01-24T22:05:38.047 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:05:38.093 INFO:teuthology.orchestra.run.smithi035.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 4.6 MB/s | 225 kB 00:00 2023-01-24T22:05:38.162 INFO:teuthology.orchestra.run.smithi035.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 54 MB/s | 3.7 MB 00:00 2023-01-24T22:05:38.245 INFO:teuthology.orchestra.run.smithi035.stdout:(3/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 57 MB/s | 4.7 MB 00:00 2023-01-24T22:05:38.360 INFO:teuthology.orchestra.run.smithi035.stdout:(4/8): python3-scipy-1.0.0-20.module+el8.1.0+33 46 MB/s | 15 MB 00:00 2023-01-24T22:05:38.377 INFO:teuthology.orchestra.run.smithi035.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 9.7 MB/s | 169 kB 00:00 2023-01-24T22:05:38.399 INFO:teuthology.orchestra.run.smithi035.stdout:(6/8): libgfortran-8.4.1-1.el8.x86_64.rpm 29 MB/s | 639 kB 00:00 2023-01-24T22:05:38.416 INFO:teuthology.orchestra.run.smithi035.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 27 MB/s | 4.6 MB 00:00 2023-01-24T22:05:38.519 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T22:05:38.724 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T22:05:38.862 INFO:teuthology.orchestra.run.smithi035.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 9.1 MB/s | 7.4 MB 00:00 2023-01-24T22:05:38.863 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:38.864 INFO:teuthology.orchestra.run.smithi035.stdout:Total 44 MB/s | 36 MB 00:00 2023-01-24T22:05:38.874 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:05:38.895 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:05:38.895 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:05:39.024 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T22:05:39.024 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T22:05:39.024 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-01-24T22:05:39.025 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-01-24T22:05:39.025 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-01-24T22:05:39.025 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-01-24T22:05:39.025 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-01-24T22:05:39.025 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-01-24T22:05:39.346 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:05:39.350 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:05:39.989 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:05:40.032 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-01-24T22:05:40.033 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:21 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:40.181 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T22:05:40.327 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T22:05:40.464 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T22:05:41.034 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T22:05:41.223 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T22:05:41.768 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T22:05:41.943 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:05:41.981 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T22:05:42.041 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:42.041 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:05:42.041 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:42.042 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T22:05:42.042 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T22:05:42.042 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T22:05:42.042 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T22:05:42.042 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T22:05:42.043 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T22:05:42.043 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T22:05:42.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:42.043 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:42.212 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-rook 2023-01-24T22:05:42.570 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:42.741 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T22:05:43.079 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T22:05:43.322 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T22:05:43.432 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:24 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:45.217 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:45.219 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================================= 2023-01-24T22:05:45.219 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:05:45.219 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================================= 2023-01-24T22:05:45.220 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:05:45.220 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T22:05:45.220 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:05:45.220 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T22:05:45.220 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T22:05:45.221 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T22:05:45.221 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T22:05:45.221 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T22:05:45.221 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T22:05:45.221 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T22:05:45.222 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T22:05:45.222 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T22:05:45.222 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T22:05:45.222 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T22:05:45.222 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T22:05:45.223 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:45.223 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:05:45.223 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================================= 2023-01-24T22:05:45.223 INFO:teuthology.orchestra.run.smithi072.stdout:Install 13 Packages 2023-01-24T22:05:45.223 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:45.224 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 1.9 M 2023-01-24T22:05:45.224 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 25 M 2023-01-24T22:05:45.225 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:45.304 INFO:teuthology.orchestra.run.smithi072.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 399 kB/s | 33 kB 00:00 2023-01-24T22:05:45.306 INFO:teuthology.orchestra.run.smithi072.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 193 kB/s | 16 kB 00:00 2023-01-24T22:05:45.360 INFO:teuthology.orchestra.run.smithi072.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.5 MB/s | 81 kB 00:00 2023-01-24T22:05:45.367 INFO:teuthology.orchestra.run.smithi072.stdout:(4/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 324 kB/s | 47 kB 00:00 2023-01-24T22:05:45.387 INFO:teuthology.orchestra.run.smithi072.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 2.2 MB/s | 61 kB 00:00 2023-01-24T22:05:45.415 INFO:teuthology.orchestra.run.smithi072.stdout:(6/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 962 kB/s | 27 kB 00:00 2023-01-24T22:05:45.427 INFO:teuthology.orchestra.run.smithi072.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.8 MB/s | 126 kB 00:00 2023-01-24T22:05:45.440 INFO:teuthology.orchestra.run.smithi072.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 8.4 MB/s | 110 kB 00:00 2023-01-24T22:05:45.453 INFO:teuthology.orchestra.run.smithi072.stdout:(9/13): python3-jsonpointer-1.10-11.el8.noarch. 1.6 MB/s | 20 kB 00:00 2023-01-24T22:05:45.466 INFO:teuthology.orchestra.run.smithi072.stdout:(10/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 11 MB/s | 155 kB 00:00 2023-01-24T22:05:45.473 INFO:teuthology.orchestra.run.smithi072.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 578 kB/s | 61 kB 00:00 2023-01-24T22:05:45.478 INFO:teuthology.orchestra.run.smithi072.stdout:(12/13): python3-requests-oauthlib-1.0.0-1.el8. 3.5 MB/s | 43 kB 00:00 2023-01-24T22:05:45.503 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T22:05:45.589 INFO:teuthology.orchestra.run.smithi072.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 4.1 MB/s | 1.2 MB 00:00 2023-01-24T22:05:45.589 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:45.590 INFO:teuthology.orchestra.run.smithi072.stdout:Total 5.2 MB/s | 1.9 MB 00:00 2023-01-24T22:05:45.594 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:45.608 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:45.609 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:45.696 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:45.700 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:46.180 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:46.411 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T22:05:46.655 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T22:05:46.866 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T22:05:46.935 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T22:05:47.013 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T22:05:47.247 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-01-24T22:05:47.342 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T22:05:47.342 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T22:05:47.342 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-01-24T22:05:47.343 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-01-24T22:05:47.343 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-01-24T22:05:47.343 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-01-24T22:05:47.344 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-01-24T22:05:47.344 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-01-24T22:05:47.497 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-01-24T22:05:47.715 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-01-24T22:05:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-01-24T22:05:48.226 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T22:05:48.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T22:05:48.456 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-01-24T22:05:48.457 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:29 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:48.682 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T22:05:49.858 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T22:05:50.151 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T22:05:50.348 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T22:05:50.494 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:05:50.604 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:50.604 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:05:50.604 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:50.604 INFO:teuthology.orchestra.run.smithi035.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T22:05:50.605 INFO:teuthology.orchestra.run.smithi035.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T22:05:50.605 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T22:05:50.605 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T22:05:50.605 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T22:05:50.605 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T22:05:50.606 INFO:teuthology.orchestra.run.smithi035.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T22:05:50.606 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:50.606 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:05:50.611 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T22:05:50.611 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T22:05:50.611 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T22:05:50.611 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T22:05:50.611 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T22:05:50.612 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T22:05:50.612 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T22:05:50.612 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T22:05:50.612 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-01-24T22:05:50.613 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-01-24T22:05:50.613 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-24T22:05:50.613 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T22:05:50.613 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-01-24T22:05:50.857 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-rook 2023-01-24T22:05:51.238 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:51.684 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-01-24T22:05:51.684 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:32 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:52.175 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:33 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:53.556 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:05:53.654 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:53.655 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:05:53.655 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:05:53.655 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T22:05:53.656 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T22:05:53.656 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T22:05:53.656 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T22:05:53.656 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T22:05:53.656 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T22:05:53.657 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T22:05:53.657 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T22:05:53.657 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T22:05:53.657 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T22:05:53.658 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T22:05:53.658 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T22:05:53.658 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:53.658 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:05:53.800 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T22:05:54.052 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:05:54.054 INFO:teuthology.orchestra.run.smithi035.stdout:======================================================================================================= 2023-01-24T22:05:54.054 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:05:54.054 INFO:teuthology.orchestra.run.smithi035.stdout:======================================================================================================= 2023-01-24T22:05:54.055 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:05:54.055 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T22:05:54.055 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:05:54.055 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T22:05:54.056 INFO:teuthology.orchestra.run.smithi035.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T22:05:54.056 INFO:teuthology.orchestra.run.smithi035.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T22:05:54.056 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T22:05:54.056 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T22:05:54.056 INFO:teuthology.orchestra.run.smithi035.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T22:05:54.057 INFO:teuthology.orchestra.run.smithi035.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T22:05:54.057 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T22:05:54.057 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T22:05:54.057 INFO:teuthology.orchestra.run.smithi035.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T22:05:54.058 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T22:05:54.058 INFO:teuthology.orchestra.run.smithi035.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T22:05:54.058 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:54.058 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:05:54.058 INFO:teuthology.orchestra.run.smithi035.stdout:======================================================================================================= 2023-01-24T22:05:54.059 INFO:teuthology.orchestra.run.smithi035.stdout:Install 13 Packages 2023-01-24T22:05:54.059 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:05:54.059 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 1.9 M 2023-01-24T22:05:54.060 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 25 M 2023-01-24T22:05:54.060 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:05:54.144 INFO:teuthology.orchestra.run.smithi035.stdout:(1/13): python3-cachetools-3.1.1-4.el8.noarch.r 382 kB/s | 33 kB 00:00 2023-01-24T22:05:54.144 INFO:teuthology.orchestra.run.smithi035.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 189 kB/s | 16 kB 00:00 2023-01-24T22:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:05:54.203 INFO:teuthology.orchestra.run.smithi035.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.3 MB/s | 81 kB 00:00 2023-01-24T22:05:54.207 INFO:teuthology.orchestra.run.smithi035.stdout:(4/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 314 kB/s | 47 kB 00:00 2023-01-24T22:05:54.231 INFO:teuthology.orchestra.run.smithi035.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 2.1 MB/s | 61 kB 00:00 2023-01-24T22:05:54.265 INFO:teuthology.orchestra.run.smithi035.stdout:(6/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 773 kB/s | 27 kB 00:00 2023-01-24T22:05:54.282 INFO:teuthology.orchestra.run.smithi035.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.7 MB/s | 126 kB 00:00 2023-01-24T22:05:54.295 INFO:teuthology.orchestra.run.smithi035.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 8.1 MB/s | 110 kB 00:00 2023-01-24T22:05:54.307 INFO:teuthology.orchestra.run.smithi035.stdout:(9/13): python3-jsonpointer-1.10-11.el8.noarch. 1.7 MB/s | 20 kB 00:00 2023-01-24T22:05:54.319 INFO:teuthology.orchestra.run.smithi035.stdout:(10/13): python3-websocket-client-0.56.0-5.el8. 549 kB/s | 61 kB 00:00 2023-01-24T22:05:54.321 INFO:teuthology.orchestra.run.smithi035.stdout:(11/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 11 MB/s | 155 kB 00:00 2023-01-24T22:05:54.348 INFO:teuthology.orchestra.run.smithi035.stdout:(12/13): python3-requests-oauthlib-1.0.0-1.el8. 1.4 MB/s | 43 kB 00:00 2023-01-24T22:05:54.541 INFO:teuthology.orchestra.run.smithi035.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.9 MB/s | 1.2 MB 00:00 2023-01-24T22:05:54.542 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:54.542 INFO:teuthology.orchestra.run.smithi035.stdout:Total 4.0 MB/s | 1.9 MB 00:00 2023-01-24T22:05:54.548 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:05:54.563 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:05:54.564 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:05:54.657 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:05:54.662 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:05:55.001 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:05:55.185 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:05:55.453 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T22:05:55.748 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T22:05:56.000 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T22:05:56.287 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-01-24T22:05:56.520 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-01-24T22:05:56.688 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-01-24T22:05:56.807 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:05:56.808 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================== 2023-01-24T22:05:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:05:56.808 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================== 2023-01-24T22:05:56.809 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:05:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T22:05:56.810 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:05:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T22:05:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T22:05:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T22:05:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T22:05:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:56.812 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:05:56.812 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================== 2023-01-24T22:05:56.812 INFO:teuthology.orchestra.run.smithi072.stdout:Install 5 Packages 2023-01-24T22:05:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:05:56.814 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 5.9 M 2023-01-24T22:05:56.814 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 26 M 2023-01-24T22:05:56.815 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:05:56.880 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-01-24T22:05:56.919 INFO:teuthology.orchestra.run.smithi072.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 550 kB/s | 60 kB 00:00 2023-01-24T22:05:56.988 INFO:teuthology.orchestra.run.smithi072.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 647 kB/s | 115 kB 00:00 2023-01-24T22:05:57.007 INFO:teuthology.orchestra.run.smithi072.stdout:(3/5): python3-babel-2.5.1-5.el8.noarch.rpm 54 MB/s | 4.8 MB 00:00 2023-01-24T22:05:57.030 INFO:teuthology.orchestra.run.smithi072.stdout:(4/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 12 MB/s | 538 kB 00:00 2023-01-24T22:05:57.055 INFO:teuthology.orchestra.run.smithi072.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.9 MB/s | 476 kB 00:00 2023-01-24T22:05:57.056 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:05:57.057 INFO:teuthology.orchestra.run.smithi072.stdout:Total 24 MB/s | 5.9 MB 00:00 2023-01-24T22:05:57.060 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:05:57.069 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:05:57.070 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:05:57.132 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T22:05:57.176 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:05:57.181 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:05:57.384 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T22:05:57.674 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T22:05:58.202 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:05:58.535 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T22:05:58.792 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T22:05:58.943 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T22:05:59.154 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T22:05:59.233 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T22:05:59.378 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T22:05:59.409 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T22:05:59.613 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T22:05:59.656 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T22:05:59.656 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T22:05:59.656 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T22:05:59.656 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T22:05:59.656 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T22:05:59.657 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T22:05:59.657 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T22:05:59.657 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T22:05:59.657 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-01-24T22:05:59.657 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-01-24T22:05:59.658 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-24T22:05:59.658 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T22:05:59.658 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-01-24T22:05:59.870 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T22:05:59.870 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T22:05:59.870 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T22:05:59.870 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T22:05:59.871 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T22:06:00.807 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-01-24T22:06:00.807 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:41 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:00.867 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T22:06:00.867 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:41 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:02.878 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:06:02.930 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:02.977 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:02.977 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:06:02.978 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:06:02.978 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T22:06:02.978 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T22:06:02.978 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T22:06:02.978 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T22:06:02.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:02.979 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:03.038 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:03.038 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:03.038 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:06:03.038 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T22:06:03.039 INFO:teuthology.orchestra.run.smithi035.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T22:06:03.039 INFO:teuthology.orchestra.run.smithi035.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T22:06:03.039 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T22:06:03.039 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T22:06:03.039 INFO:teuthology.orchestra.run.smithi035.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T22:06:03.040 INFO:teuthology.orchestra.run.smithi035.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T22:06:03.040 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T22:06:03.040 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T22:06:03.040 INFO:teuthology.orchestra.run.smithi035.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T22:06:03.040 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T22:06:03.041 INFO:teuthology.orchestra.run.smithi035.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T22:06:03.041 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:03.041 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:03.117 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-fuse 2023-01-24T22:06:03.169 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T22:06:03.482 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:03.557 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:04.393 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:45 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:04.458 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:45 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:06.204 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:06.205 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:06.205 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:06:06.205 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:06.205 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:06:06.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse 2023-01-24T22:06:06.206 INFO:teuthology.orchestra.run.smithi072.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T22:06:06.206 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-01-24T22:06:06.206 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T22:06:06.206 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:06.206 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:06:06.207 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:06.207 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2023-01-24T22:06:06.207 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:06.208 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 954 k 2023-01-24T22:06:06.208 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 3.0 M 2023-01-24T22:06:06.208 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:06:06.234 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.9 MB/s | 83 kB 00:00 2023-01-24T22:06:06.580 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:06.581 INFO:teuthology.orchestra.run.smithi035.stdout:============================================================================================== 2023-01-24T22:06:06.582 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:06:06.582 INFO:teuthology.orchestra.run.smithi035.stdout:============================================================================================== 2023-01-24T22:06:06.582 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:06:06.582 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T22:06:06.582 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:06:06.583 INFO:teuthology.orchestra.run.smithi035.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T22:06:06.583 INFO:teuthology.orchestra.run.smithi035.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T22:06:06.583 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T22:06:06.583 INFO:teuthology.orchestra.run.smithi035.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T22:06:06.584 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:06.584 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:06:06.584 INFO:teuthology.orchestra.run.smithi035.stdout:============================================================================================== 2023-01-24T22:06:06.584 INFO:teuthology.orchestra.run.smithi035.stdout:Install 5 Packages 2023-01-24T22:06:06.584 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:06.585 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 5.9 M 2023-01-24T22:06:06.585 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 26 M 2023-01-24T22:06:06.585 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:06:06.693 INFO:teuthology.orchestra.run.smithi035.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 553 kB/s | 60 kB 00:00 2023-01-24T22:06:06.787 INFO:teuthology.orchestra.run.smithi035.stdout:(2/5): python3-babel-2.5.1-5.el8.noarch.rpm 52 MB/s | 4.8 MB 00:00 2023-01-24T22:06:06.804 INFO:teuthology.orchestra.run.smithi035.stdout:(3/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 28 MB/s | 538 kB 00:00 2023-01-24T22:06:06.837 INFO:teuthology.orchestra.run.smithi035.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2023-01-24T22:06:07.023 INFO:teuthology.orchestra.run.smithi035.stdout:(5/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 262 kB/s | 115 kB 00:00 2023-01-24T22:06:07.024 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:07.025 INFO:teuthology.orchestra.run.smithi035.stdout:Total 13 MB/s | 5.9 MB 00:00 2023-01-24T22:06:07.028 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:06:07.039 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:06:07.039 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:06:07.158 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:06:07.163 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:06:07.388 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 738 kB/s | 872 kB 00:01 2023-01-24T22:06:07.389 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:07.390 INFO:teuthology.orchestra.run.smithi072.stdout:Total 806 kB/s | 954 kB 00:01 2023-01-24T22:06:07.391 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:06:07.415 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:06:07.415 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:06:07.504 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:06:07.508 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:06:08.242 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:06:08.495 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:06:08.608 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T22:06:08.875 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T22:06:09.231 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T22:06:09.534 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T22:06:09.727 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T22:06:10.026 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T22:06:10.026 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T22:06:10.026 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T22:06:10.026 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T22:06:10.027 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T22:06:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T22:06:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T22:06:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T22:06:11.079 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T22:06:11.158 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T22:06:11.159 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:52 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:12.094 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T22:06:12.095 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:53 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:13.072 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:13.180 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:13.180 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:13.180 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:06:13.180 INFO:teuthology.orchestra.run.smithi035.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T22:06:13.181 INFO:teuthology.orchestra.run.smithi035.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T22:06:13.181 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T22:06:13.181 INFO:teuthology.orchestra.run.smithi035.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T22:06:13.181 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:13.181 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:13.296 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-fuse 2023-01-24T22:06:13.685 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:14.011 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:06:14.106 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:14.107 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:06:14.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T22:06:14.107 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:14.107 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:14.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-volume 2023-01-24T22:06:14.607 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:14.676 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:02:55 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:15.454 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:56 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:16.564 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:16.565 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:16.565 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:06:16.566 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:16.566 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:06:16.566 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse 2023-01-24T22:06:16.566 INFO:teuthology.orchestra.run.smithi035.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T22:06:16.566 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2023-01-24T22:06:16.567 INFO:teuthology.orchestra.run.smithi035.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T22:06:16.567 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:16.567 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:06:16.567 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:16.567 INFO:teuthology.orchestra.run.smithi035.stdout:Install 2 Packages 2023-01-24T22:06:16.568 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:16.568 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 954 k 2023-01-24T22:06:16.568 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 3.0 M 2023-01-24T22:06:16.569 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:06:16.597 INFO:teuthology.orchestra.run.smithi035.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.6 MB/s | 83 kB 00:00 2023-01-24T22:06:16.803 INFO:teuthology.orchestra.run.smithi035.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 872 kB 00:00 2023-01-24T22:06:16.803 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:16.804 INFO:teuthology.orchestra.run.smithi035.stdout:Total 3.9 MB/s | 954 kB 00:00 2023-01-24T22:06:16.805 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:06:16.828 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:06:16.829 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:06:16.919 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:06:16.923 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:06:17.100 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:06:17.277 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:17.278 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:17.278 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-01-24T22:06:17.278 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:17.278 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:06:17.278 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T22:06:17.279 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:17.279 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:06:17.279 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:17.279 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:06:17.279 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:17.280 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 263 k 2023-01-24T22:06:17.280 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 1.3 M 2023-01-24T22:06:17.280 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:06:17.466 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.4 MB/s | 263 kB 00:00 2023-01-24T22:06:17.467 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:17.467 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.4 MB/s | 263 kB 00:00 2023-01-24T22:06:17.468 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:06:17.478 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:06:17.478 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:06:17.500 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:06:17.504 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:06:17.682 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:06:17.953 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:18.336 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:18.527 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T22:06:18.919 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T22:06:19.336 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T22:06:19.337 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T22:06:19.354 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:19.354 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:00 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:20.476 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T22:06:20.476 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:01 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:21.299 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:06:21.398 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:06:21.399 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:06:21.399 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:21.517 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados-devel 2023-01-24T22:06:21.886 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:22.471 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:22.579 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:22.579 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:22.580 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T22:06:22.580 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:22.580 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:22.701 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-volume 2023-01-24T22:06:22.772 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:03 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:23.093 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:23.964 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:04 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:24.552 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:24.553 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:24.553 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-01-24T22:06:24.553 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:24.554 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:06:24.554 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T22:06:24.554 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:24.554 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:06:24.554 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:24.555 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:06:24.555 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:24.555 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 130 k 2023-01-24T22:06:24.555 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 464 k 2023-01-24T22:06:24.555 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:06:24.720 INFO:teuthology.orchestra.run.smithi072.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 783 kB/s | 130 kB 00:00 2023-01-24T22:06:24.721 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:24.721 INFO:teuthology.orchestra.run.smithi072.stdout:Total 776 kB/s | 130 kB 00:00 2023-01-24T22:06:24.721 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:06:24.727 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:06:24.728 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:06:24.783 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:06:24.788 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:06:24.912 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:06:25.076 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:25.351 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:25.886 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:25.887 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:25.888 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2023-01-24T22:06:25.888 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:25.888 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:06:25.888 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T22:06:25.889 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:25.889 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:06:25.889 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:25.889 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:06:25.889 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:25.890 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 263 k 2023-01-24T22:06:25.890 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 1.3 M 2023-01-24T22:06:25.890 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:06:26.077 INFO:teuthology.orchestra.run.smithi035.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.4 MB/s | 263 kB 00:00 2023-01-24T22:06:26.078 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:26.079 INFO:teuthology.orchestra.run.smithi035.stdout:Total 1.3 MB/s | 263 kB 00:00 2023-01-24T22:06:26.079 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:06:26.090 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:06:26.090 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:06:26.118 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:06:26.122 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:06:26.309 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:06:26.380 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:26.381 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:07 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:26.642 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:27.075 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:28.154 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T22:06:28.154 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:09 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:28.621 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:06:28.716 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:28.717 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:06:28.717 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:06:28.717 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:28.717 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:28.851 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs2 2023-01-24T22:06:29.216 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:11 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:30.401 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:30.507 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:30.507 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:30.507 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T22:06:30.508 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:30.508 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:30.643 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install librados-devel 2023-01-24T22:06:31.037 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:31.870 INFO:teuthology.orchestra.run.smithi072.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:31.945 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:12 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:32.319 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:32.320 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:32.321 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:32.428 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs-devel 2023-01-24T22:06:32.794 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:33.647 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:14 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:33.887 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:33.888 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:33.888 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repo Size 2023-01-24T22:06:33.889 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:33.889 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:06:33.889 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T22:06:33.889 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:33.890 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:06:33.890 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:33.890 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:06:33.890 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:33.891 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 130 k 2023-01-24T22:06:33.891 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 464 k 2023-01-24T22:06:33.891 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:06:34.223 INFO:teuthology.orchestra.run.smithi035.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 388 kB/s | 130 kB 00:00 2023-01-24T22:06:34.224 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:34.225 INFO:teuthology.orchestra.run.smithi035.stdout:Total 387 kB/s | 130 kB 00:00 2023-01-24T22:06:34.225 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:06:34.232 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:06:34.232 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:06:34.294 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:06:34.298 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:06:34.547 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:06:34.724 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:35.016 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:35.443 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:35.444 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:35.445 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-01-24T22:06:35.445 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:35.445 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:06:35.445 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T22:06:35.446 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:35.446 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:06:35.446 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:06:35.446 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:06:35.446 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:35.447 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 25 k 2023-01-24T22:06:35.447 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 109 k 2023-01-24T22:06:35.447 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:06:35.715 INFO:teuthology.orchestra.run.smithi072.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 91 kB/s | 25 kB 00:00 2023-01-24T22:06:35.716 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:35.716 INFO:teuthology.orchestra.run.smithi072.stdout:Total 91 kB/s | 25 kB 00:00 2023-01-24T22:06:35.716 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:06:35.720 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:06:35.720 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:06:35.721 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:06:35.726 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:06:35.782 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:06:35.933 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:36.125 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:36.126 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:17 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:36.179 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:37.127 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:37.127 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:18 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:38.141 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:38.249 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:38.249 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:38.249 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:06:38.249 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:38.250 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:38.372 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install libcephfs2 2023-01-24T22:06:38.765 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:38.977 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:06:39.072 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:39.073 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:06:39.073 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:06:39.073 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:06:39.074 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:39.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados2 2023-01-24T22:06:39.546 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:39.647 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:20 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:40.383 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:21 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:41.063 INFO:teuthology.orchestra.run.smithi035.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:41.545 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:41.546 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:06:41.547 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:41.638 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install libcephfs-devel 2023-01-24T22:06:41.758 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:42.029 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:42.209 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:42.210 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:42.210 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:42.314 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librbd1 2023-01-24T22:06:42.679 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:42.903 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:23 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:43.508 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:24 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:44.876 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:44.900 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:44.901 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:44.901 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repo Size 2023-01-24T22:06:44.902 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:44.902 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:06:44.903 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T22:06:44.903 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:44.903 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:06:44.904 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:06:44.904 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:06:44.904 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:44.905 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 25 k 2023-01-24T22:06:44.905 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 109 k 2023-01-24T22:06:44.906 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:06:45.022 INFO:teuthology.orchestra.run.smithi035.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 205 kB/s | 25 kB 00:00 2023-01-24T22:06:45.023 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:06:45.024 INFO:teuthology.orchestra.run.smithi035.stdout:Total 203 kB/s | 25 kB 00:00 2023-01-24T22:06:45.024 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:06:45.027 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:06:45.032 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:06:45.033 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:06:45.034 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:06:45.088 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:06:45.281 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:45.333 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:45.334 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:45.335 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:45.434 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rados 2023-01-24T22:06:45.535 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:45.803 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:46.537 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:06:46.538 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:27 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:46.646 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:27 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:48.041 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:48.506 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:48.507 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:48.508 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:48.524 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:06:48.616 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rgw 2023-01-24T22:06:48.631 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:48.631 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:06:48.631 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:06:48.632 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:06:48.632 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:48.762 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install librados2 2023-01-24T22:06:48.981 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:49.153 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:49.872 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:30 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:50.015 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:31 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:51.241 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:51.492 INFO:teuthology.orchestra.run.smithi035.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:51.687 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:51.688 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:51.688 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:51.792 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-cephfs 2023-01-24T22:06:51.978 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:51.979 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:06:51.980 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:52.088 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install librbd1 2023-01-24T22:06:52.158 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:52.480 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:53.022 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:34 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:53.348 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:34 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:54.415 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:54.745 INFO:teuthology.orchestra.run.smithi035.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:54.870 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:54.872 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:54.872 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:54.984 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rbd 2023-01-24T22:06:55.233 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:55.235 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:06:55.235 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:55.334 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install python3-rados 2023-01-24T22:06:55.346 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:55.724 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:56.182 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:37 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:56.600 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:37 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:57.532 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:57.991 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:06:57.992 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-01-24T22:06:57.992 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:06:58.027 INFO:teuthology.orchestra.run.smithi035.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:06:58.099 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-fuse 2023-01-24T22:06:58.463 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:58.503 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:06:58.504 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:06:58.504 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:06:58.622 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install python3-rgw 2023-01-24T22:06:59.013 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:06:59.566 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:40 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:06:59.938 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:40 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:01.350 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:07:01.351 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:01.352 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-01-24T22:07:01.352 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:01.352 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:07:01.352 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T22:07:01.352 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:01.353 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:07:01.353 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:01.353 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:07:01.353 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:01.353 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 90 k 2023-01-24T22:07:01.354 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 264 k 2023-01-24T22:07:01.354 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:07:01.427 INFO:teuthology.orchestra.run.smithi035.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:07:01.705 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 255 kB/s | 90 kB 00:00 2023-01-24T22:07:01.706 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:01.707 INFO:teuthology.orchestra.run.smithi072.stdout:Total 254 kB/s | 90 kB 00:00 2023-01-24T22:07:01.707 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:07:01.718 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:07:01.719 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:07:01.774 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:07:01.779 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:07:01.887 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:07:01.907 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:01.908 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:07:01.908 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:02.014 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install python3-cephfs 2023-01-24T22:07:02.081 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:02.349 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:02.406 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:03.293 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:44 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:03.326 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:03.326 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:44 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:04.727 INFO:teuthology.orchestra.run.smithi035.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:07:05.189 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:07:05.212 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:05.213 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:07:05.214 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:05.286 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:07:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:05.286 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:07:05.316 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install python3-rbd 2023-01-24T22:07:05.405 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-mirror 2023-01-24T22:07:05.707 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:05.771 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:06.617 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:47 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:06.623 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:47 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:08.118 INFO:teuthology.orchestra.run.smithi035.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T22:07:08.402 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:07:08.403 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:08.403 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-01-24T22:07:08.403 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:08.403 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:07:08.404 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T22:07:08.404 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:08.404 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:07:08.404 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:08.405 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:07:08.405 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:08.405 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 3.5 M 2023-01-24T22:07:08.405 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 16 M 2023-01-24T22:07:08.406 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:07:08.600 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:08.601 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2023-01-24T22:07:08.601 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:08.700 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install rbd-fuse 2023-01-24T22:07:09.091 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:09.358 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 3.6 MB/s | 3.5 MB 00:00 2023-01-24T22:07:09.359 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:09.359 INFO:teuthology.orchestra.run.smithi072.stdout:Total 3.6 MB/s | 3.5 MB 00:00 2023-01-24T22:07:09.360 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:07:09.373 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:07:09.373 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:07:09.441 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:07:09.445 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:07:09.884 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:07:10.000 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:50 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:10.472 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:11.472 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:11.472 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:52 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:11.974 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:11.975 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:11.975 INFO:teuthology.orchestra.run.smithi035.stdout: Package Architecture Version Repository Size 2023-01-24T22:07:11.975 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:11.976 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:07:11.976 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T22:07:11.976 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:11.976 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:07:11.976 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:11.977 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:07:11.977 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:11.977 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 90 k 2023-01-24T22:07:11.977 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 264 k 2023-01-24T22:07:11.977 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:07:12.232 INFO:teuthology.orchestra.run.smithi035.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 353 kB/s | 90 kB 00:00 2023-01-24T22:07:12.233 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:12.233 INFO:teuthology.orchestra.run.smithi035.stdout:Total 351 kB/s | 90 kB 00:00 2023-01-24T22:07:12.233 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:07:12.246 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:07:12.246 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:07:12.306 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:07:12.310 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:07:12.431 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:07:12.612 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:12.873 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:13.321 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:07:13.416 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:13.417 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:07:13.417 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:13.417 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:13.417 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:07:13.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-nbd 2023-01-24T22:07:13.912 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:13.913 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:54 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:13.954 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:14.791 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:55 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:15.830 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:07:15.937 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:15.937 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:07:15.937 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:15.937 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:15.938 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:16.059 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install rbd-mirror 2023-01-24T22:07:16.449 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:16.571 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-01-24T22:07:16.572 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-01-24T22:07:16.573 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:16.573 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-01-24T22:07:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T22:07:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:16.573 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-01-24T22:07:16.574 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-01-24T22:07:16.574 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-01-24T22:07:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:16.574 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 172 k 2023-01-24T22:07:16.574 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 548 k 2023-01-24T22:07:16.575 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-01-24T22:07:16.969 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 435 kB/s | 172 kB 00:00 2023-01-24T22:07:16.970 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:16.971 INFO:teuthology.orchestra.run.smithi072.stdout:Total 433 kB/s | 172 kB 00:00 2023-01-24T22:07:16.971 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-01-24T22:07:16.984 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-01-24T22:07:16.984 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-01-24T22:07:17.040 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-01-24T22:07:17.044 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-01-24T22:07:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-01-24T22:07:17.326 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:03:58 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:17.345 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:17.619 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:18.863 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:18.863 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:59 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:19.267 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:19.267 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:19.267 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repo Size 2023-01-24T22:07:19.268 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:19.268 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:07:19.268 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T22:07:19.268 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:19.268 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:07:19.269 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:19.269 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:07:19.269 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:19.270 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 3.5 M 2023-01-24T22:07:19.270 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 16 M 2023-01-24T22:07:19.270 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:07:20.476 INFO:teuthology.orchestra.run.smithi035.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 2.9 MB/s | 3.5 MB 00:01 2023-01-24T22:07:20.477 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:20.477 INFO:teuthology.orchestra.run.smithi035.stdout:Total 2.9 MB/s | 3.5 MB 00:01 2023-01-24T22:07:20.478 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:07:20.493 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:07:20.493 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:07:20.568 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:07:20.572 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:07:20.712 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-01-24T22:07:20.810 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:20.810 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-01-24T22:07:20.810 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:20.811 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:20.811 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-01-24T22:07:20.926 DEBUG:teuthology.parallel:result is None 2023-01-24T22:07:21.050 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:07:21.235 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:21.705 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:22.758 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:22.759 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:04:03 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:24.684 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:07:24.791 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:24.792 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:07:24.792 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:24.792 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:24.792 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:24.905 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install rbd-nbd 2023-01-24T22:07:25.299 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2023-01-24T22:07:26.202 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:04:07 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:28.121 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2023-01-24T22:07:28.122 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:28.122 INFO:teuthology.orchestra.run.smithi035.stdout: Package Architecture Version Repository Size 2023-01-24T22:07:28.123 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:28.123 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2023-01-24T22:07:28.123 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T22:07:28.123 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:28.123 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2023-01-24T22:07:28.124 INFO:teuthology.orchestra.run.smithi035.stdout:================================================================================ 2023-01-24T22:07:28.124 INFO:teuthology.orchestra.run.smithi035.stdout:Install 1 Package 2023-01-24T22:07:28.124 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:28.124 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 172 k 2023-01-24T22:07:28.125 INFO:teuthology.orchestra.run.smithi035.stdout:Installed size: 548 k 2023-01-24T22:07:28.125 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2023-01-24T22:07:28.397 INFO:teuthology.orchestra.run.smithi035.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 628 kB/s | 172 kB 00:00 2023-01-24T22:07:28.398 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2023-01-24T22:07:28.399 INFO:teuthology.orchestra.run.smithi035.stdout:Total 625 kB/s | 172 kB 00:00 2023-01-24T22:07:28.399 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2023-01-24T22:07:28.412 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2023-01-24T22:07:28.412 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2023-01-24T22:07:28.472 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2023-01-24T22:07:28.477 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2023-01-24T22:07:28.603 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2023-01-24T22:07:28.784 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:29.083 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:30.180 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T22:07:30.180 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:04:11 ago on Tue 24 Jan 2023 10:03:19 PM UTC. 2023-01-24T22:07:32.116 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2023-01-24T22:07:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:32.224 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2023-01-24T22:07:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T22:07:32.225 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:32.225 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2023-01-24T22:07:32.344 DEBUG:teuthology.parallel:result is None 2023-01-24T22:07:32.345 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2023-01-24T22:07:32.384 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2023-01-24T22:07:32.385 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:07:32.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T22:07:32.532 DEBUG:teuthology.orchestra.run.smithi035:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T22:07:32.552 INFO:teuthology.orchestra.run.smithi035.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T22:07:32.553 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T22:07:32.554 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T22:07:32.554 DEBUG:teuthology.orchestra.run.smithi072:> sudo lsb_release -is 2023-01-24T22:07:32.601 INFO:teuthology.orchestra.run.smithi072.stdout:RedHatEnterprise 2023-01-24T22:07:32.603 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T22:07:32.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T22:07:32.751 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T22:07:32.772 INFO:teuthology.orchestra.run.smithi072.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T22:07:32.772 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T22:07:32.773 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T22:07:32.773 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T22:07:32.773 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:32.774 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T22:07:32.799 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:32.799 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T22:07:32.840 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T22:07:32.841 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:32.841 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T22:07:32.865 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T22:07:32.929 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:32.930 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T22:07:32.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T22:07:33.021 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T22:07:33.022 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:33.022 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T22:07:33.047 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T22:07:33.111 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:33.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T22:07:33.137 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T22:07:33.203 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T22:07:33.268 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T22:07:33.269 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 777 /var/log/ceph 2023-01-24T22:07:33.271 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-01-24T22:07:33.295 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T22:07:33.296 DEBUG:teuthology.orchestra.run.smithi035:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T22:07:33.335 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T22:07:33.359 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T22:07:33.360 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T22:07:33.400 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T22:07:33.435 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T22:07:33.435 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2023-01-24T22:07:33.435 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135870', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135870', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}, {'volumes': {'size': 50}}, {'machine': {'ram': 15000}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi035.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXB8lkoP5CVWO87fZakSfTl3CmucUtKV6//WjBPrCsmqczBhaoxe3wq87zYcyxh4IAgazYhhDvZNIS0Qb2qNqBC9q/CQXy0SGGcR/mONTNu9O25S0qy0d7qUaHZcDH4FJlk4Qh/iWef7PoWlWYn3ReEeoDmVYDqaZjZhmvpyPkR5FV50+xQHiLMYhk5+Zs2a2eIkLeRYtaQeysQBxyTU8jovWXidWaYuXicFNiD8x7so4R/Lg3A7gHqqr6gH9yRsVbcQjCoJLz0ysPIs0NiL9+RS2RoKgAA1jLxt+usn58GVYK16MuTsBUnBZp2j7HkQ9y7DWUpL8nvyyDbLfZNW4KfU/yIJmZqbGRBsY4msUkdJ62hFAwjjdBnp3cJJRYBietufP+80Wim2Y0BvcPCFiH6GTq5tqYGPRF553hdM0e3fiVhXTMaXi0eDygtb9Wo2wugGwsIkDkEwU0rlSnOcbRjkKL4k6VGKFr+sbE45yq3cysy+cZcCsc7yjJypQfxz8=', 'smithi072.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNgA5sXxsUsFFCbAae2j4eDx5EVOm3LwRR2LesAxbpVDE8bRYLN3YxpE0uDkyvGlej2AFjtFlTrzIcJ34mmzfkD3CdftcT7a3yssXhLk4snX2fLDKlAjaIrb0pcEzvFMuaSWtklGKSLQ+W7QROfgeoWjHLj/mAbcFe/IvJxfr4loPumu+PgOofGkpl/XiFUe7AOee2J/V3FViejng35ECtEJHeNcIScigh2aM4tyzuqoTmoDNv35e6qjLdqT89F1jbXQiGdVq9zEnXAaY1a2DkyBV4z7LoiCQ12JnLuqG3Q6kEQpOWfXlRErlPdAGm4K375Kq49BVbTqJR7mmAcM23TYQ+8KtPX5GMtxWtVwvBlsdP8P95dXKyDZfU7Q25VXoSRUoh2PAwBzynsNxB0WDofS2QcRcYxk5CwqlXNT2IFhhoZUDsLwtDNUV37+Fs2wgbHcLpJGouGUa1t70WhPe/TgN65rHF9dI/kAerwvxyjk98uf2KS7otqpiYlr7Pg1M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 0', 'sudo ceph osd pool set cache min_write_recency_for_promote 0']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T22:07:33.436 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T22:07:33.464 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T22:07:33.479 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T22:07:33.518 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T22:07:33.542 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:33.543 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T22:07:33.595 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T22:07:33.595 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_1 2023-01-24T22:07:33.650 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T22:07:33.650 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:33.651 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 201609 Links: 1 2023-01-24T22:07:33.651 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:33.651 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:33.651 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2023-01-24 22:07:21.459315496 +0000 2023-01-24T22:07:33.651 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2023-01-24 22:02:36.782792307 +0000 2023-01-24T22:07:33.652 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2023-01-24 22:02:36.782792307 +0000 2023-01-24T22:07:33.652 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2023-01-24T22:07:33.652 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T22:07:33.713 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2023-01-24T22:07:33.714 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2023-01-24T22:07:33.714 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000235413 s, 2.2 MB/s 2023-01-24T22:07:33.715 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T22:07:33.771 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_2 2023-01-24T22:07:33.826 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T22:07:33.826 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:33.827 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 202462 Links: 1 2023-01-24T22:07:33.827 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:33.827 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:33.827 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2023-01-24 22:07:21.460315476 +0000 2023-01-24T22:07:33.827 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2023-01-24 22:02:37.290781973 +0000 2023-01-24T22:07:33.828 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2023-01-24 22:02:37.290781973 +0000 2023-01-24T22:07:33.828 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2023-01-24T22:07:33.828 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T22:07:33.889 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2023-01-24T22:07:33.889 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2023-01-24T22:07:33.889 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000219016 s, 2.3 MB/s 2023-01-24T22:07:33.890 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T22:07:33.945 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_3 2023-01-24T22:07:34.000 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T22:07:34.000 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.000 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 204692 Links: 1 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2023-01-24 22:07:21.460315476 +0000 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2023-01-24 22:02:37.782771965 +0000 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2023-01-24 22:02:37.782771965 +0000 2023-01-24T22:07:34.001 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2023-01-24T22:07:34.002 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T22:07:34.063 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2023-01-24T22:07:34.063 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2023-01-24T22:07:34.064 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000236921 s, 2.2 MB/s 2023-01-24T22:07:34.065 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T22:07:34.120 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_4 2023-01-24T22:07:34.175 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T22:07:34.175 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.175 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 202675 Links: 1 2023-01-24T22:07:34.175 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.175 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.176 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2023-01-24 22:07:21.460315476 +0000 2023-01-24T22:07:34.176 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2023-01-24 22:02:38.269762058 +0000 2023-01-24T22:07:34.176 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2023-01-24 22:02:38.269762058 +0000 2023-01-24T22:07:34.176 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2023-01-24T22:07:34.177 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T22:07:34.237 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2023-01-24T22:07:34.238 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2023-01-24T22:07:34.238 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000190801 s, 2.7 MB/s 2023-01-24T22:07:34.239 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T22:07:34.294 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:34.295 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:34.295 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T22:07:34.311 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T22:07:34.312 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-01-24T22:07:34.367 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T22:07:34.367 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.367 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 202255 Links: 1 2023-01-24T22:07:34.368 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.368 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.368 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-01-24 22:07:10.249539937 +0000 2023-01-24T22:07:34.368 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-01-24 22:02:27.938511073 +0000 2023-01-24T22:07:34.369 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-01-24 22:02:27.938511073 +0000 2023-01-24T22:07:34.369 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-01-24T22:07:34.369 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T22:07:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-01-24T22:07:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-01-24T22:07:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.0002659 s, 1.9 MB/s 2023-01-24T22:07:34.433 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T22:07:34.490 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-01-24T22:07:34.545 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T22:07:34.546 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.546 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 203106 Links: 1 2023-01-24T22:07:34.546 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.546 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.547 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-01-24 22:07:10.249539937 +0000 2023-01-24T22:07:34.547 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-01-24 22:02:28.450500082 +0000 2023-01-24T22:07:34.547 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-01-24 22:02:28.450500082 +0000 2023-01-24T22:07:34.547 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-01-24T22:07:34.548 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T22:07:34.611 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-01-24T22:07:34.611 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-01-24T22:07:34.611 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000229725 s, 2.2 MB/s 2023-01-24T22:07:34.612 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T22:07:34.668 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-01-24T22:07:34.723 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T22:07:34.724 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.724 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 202470 Links: 1 2023-01-24T22:07:34.724 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.724 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.725 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-01-24 22:07:10.249539937 +0000 2023-01-24T22:07:34.725 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-01-24 22:02:28.941489541 +0000 2023-01-24T22:07:34.725 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-01-24 22:02:28.941489541 +0000 2023-01-24T22:07:34.725 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-01-24T22:07:34.726 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T22:07:34.786 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-01-24T22:07:34.787 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-01-24T22:07:34.787 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.00022163 s, 2.3 MB/s 2023-01-24T22:07:34.788 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T22:07:34.844 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-01-24T22:07:34.898 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T22:07:34.898 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T22:07:34.898 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 200613 Links: 1 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-01-24 22:07:10.249539937 +0000 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-01-24 22:02:29.457478464 +0000 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-01-24 22:02:29.457478464 +0000 2023-01-24T22:07:34.899 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-01-24T22:07:34.900 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T22:07:34.960 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-01-24T22:07:34.961 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-01-24T22:07:34.961 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000198068 s, 2.6 MB/s 2023-01-24T22:07:34.962 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T22:07:35.017 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:35.018 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T22:07:35.018 INFO:tasks.ceph:Generating config... 2023-01-24T22:07:35.019 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T22:07:35.020 INFO:tasks.ceph:[osd] bdev async discard = True 2023-01-24T22:07:35.020 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-01-24T22:07:35.020 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-01-24T22:07:35.020 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T22:07:35.021 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T22:07:35.021 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T22:07:35.021 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T22:07:35.021 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T22:07:35.022 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T22:07:35.022 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T22:07:35.022 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T22:07:35.023 INFO:tasks.ceph:[osd] filestore queue throttle high multiple = 2 2023-01-24T22:07:35.023 INFO:tasks.ceph:[osd] filestore queue throttle max multiple = 10 2023-01-24T22:07:35.023 INFO:tasks.ceph:[osd] journal throttle high multiple = 2 2023-01-24T22:07:35.023 INFO:tasks.ceph:[osd] journal throttle max multiple = 10 2023-01-24T22:07:35.023 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T22:07:35.024 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T22:07:35.024 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T22:07:35.024 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T22:07:35.025 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T22:07:35.025 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T22:07:35.025 INFO:tasks.ceph:[osd] osd max backfills = 9 2023-01-24T22:07:35.025 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-24T22:07:35.026 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T22:07:35.026 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T22:07:35.026 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T22:07:35.027 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T22:07:35.027 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-24T22:07:35.027 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-24T22:07:35.027 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T22:07:35.028 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-24T22:07:35.028 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-24T22:07:35.028 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-24T22:07:35.028 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-24T22:07:35.029 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-01-24T22:07:35.029 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-24T22:07:35.029 INFO:tasks.ceph:[global] ms tcp read timeout = 5 2023-01-24T22:07:35.029 INFO:tasks.ceph:[global] ms type = async 2023-01-24T22:07:35.030 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-24T22:07:35.030 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-24T22:07:35.030 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2023-01-24T22:07:35.031 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-24T22:07:35.031 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-24T22:07:35.031 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-24T22:07:35.031 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T22:07:35.032 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T22:07:35.032 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T22:07:35.032 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T22:07:35.032 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T22:07:35.033 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T22:07:35.033 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T22:07:35.033 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T22:07:35.098 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T22:07:35.101 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T22:07:35.144 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T22:07:35.210 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.35:6789]'), ('mon.c', '[v1:172.21.15.35:6790]'), ('mon.b', '[v1:172.21.15.72:6789]')] 2023-01-24T22:07:35.210 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.35:6789],[v1:172.21.15.35:6790],[v1:172.21.15.72:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T22:07:35.211 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:35.212 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T22:07:35.265 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.35:6789]' --addv c '[v1:172.21.15.35:6790]' --addv b '[v1:172.21.15.72:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:35.348 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:35.349 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: generated fsid 9ce212a6-06d7-42fa-957c-35b0162a9d82 2023-01-24T22:07:35.349 INFO:teuthology.orchestra.run.smithi035.stdout:setting min_mon_release = octopus 2023-01-24T22:07:35.349 INFO:teuthology.orchestra.run.smithi035.stdout:epoch 0 2023-01-24T22:07:35.349 INFO:teuthology.orchestra.run.smithi035.stdout:fsid 9ce212a6-06d7-42fa-957c-35b0162a9d82 2023-01-24T22:07:35.350 INFO:teuthology.orchestra.run.smithi035.stdout:last_changed 2023-01-24T22:07:35.348249+0000 2023-01-24T22:07:35.350 INFO:teuthology.orchestra.run.smithi035.stdout:created 2023-01-24T22:07:35.348249+0000 2023-01-24T22:07:35.350 INFO:teuthology.orchestra.run.smithi035.stdout:min_mon_release 15 (octopus) 2023-01-24T22:07:35.350 INFO:teuthology.orchestra.run.smithi035.stdout:election_strategy: 3 2023-01-24T22:07:35.351 INFO:teuthology.orchestra.run.smithi035.stdout:0: v1:172.21.15.35:6789/0 mon.a 2023-01-24T22:07:35.351 INFO:teuthology.orchestra.run.smithi035.stdout:1: v1:172.21.15.72:6789/0 mon.b 2023-01-24T22:07:35.351 INFO:teuthology.orchestra.run.smithi035.stdout:2: v1:172.21.15.35:6790/0 mon.c 2023-01-24T22:07:35.351 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T22:07:35.352 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T22:07:35.407 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9ce212a6-06d7-42fa-957c-35b0162a9d82... 2023-01-24T22:07:35.408 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T22:07:35.449 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T22:07:35.490 INFO:teuthology.orchestra.run.smithi035.stdout:[global] 2023-01-24T22:07:35.491 INFO:teuthology.orchestra.run.smithi035.stdout: chdir = "" 2023-01-24T22:07:35.491 INFO:teuthology.orchestra.run.smithi035.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T22:07:35.491 INFO:teuthology.orchestra.run.smithi035.stdout: auth supported = cephx 2023-01-24T22:07:35.491 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.491 INFO:teuthology.orchestra.run.smithi035.stdout: filestore xattr use omap = true 2023-01-24T22:07:35.492 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.492 INFO:teuthology.orchestra.run.smithi035.stdout: mon clock drift allowed = 1.000 2023-01-24T22:07:35.492 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.492 INFO:teuthology.orchestra.run.smithi035.stdout: osd crush chooseleaf type = 0 2023-01-24T22:07:35.492 INFO:teuthology.orchestra.run.smithi035.stdout: auth debug = true 2023-01-24T22:07:35.493 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.493 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on old message = true 2023-01-24T22:07:35.493 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on bug = true 2023-01-24T22:07:35.493 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.493 INFO:teuthology.orchestra.run.smithi035.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: mon pg warn max object skew = 0 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.494 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default size = 2 2023-01-24T22:07:35.495 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.495 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow primary affinity = true 2023-01-24T22:07:35.495 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow pg remap = true 2023-01-24T22:07:35.496 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on legacy crush tunables = false 2023-01-24T22:07:35.496 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on crush straw calc version zero = false 2023-01-24T22:07:35.496 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on no sortbitwise = false 2023-01-24T22:07:35.496 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on osd down out interval zero = false 2023-01-24T22:07:35.496 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on too few osds = false 2023-01-24T22:07:35.497 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T22:07:35.497 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T22:07:35.497 INFO:teuthology.orchestra.run.smithi035.stdout: mon_allow_pool_size_one = true 2023-01-24T22:07:35.497 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.497 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T22:07:35.498 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.498 INFO:teuthology.orchestra.run.smithi035.stdout: osd default data pool replay window = 5 2023-01-24T22:07:35.498 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.498 INFO:teuthology.orchestra.run.smithi035.stdout: mon allow pool delete = true 2023-01-24T22:07:35.498 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: mon cluster log file level = debug 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: debug asserts on shutdown = true 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: mon health detail to clog = false 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: mon host = "[v1:172.21.15.35:6789],[v1:172.21.15.35:6790],[v1:172.21.15.72:6789]" 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: mon client directed command retry = 5 2023-01-24T22:07:35.499 INFO:teuthology.orchestra.run.smithi035.stdout: mon election default strategy = 3 2023-01-24T22:07:35.500 INFO:teuthology.orchestra.run.smithi035.stdout: ms bind msgr2 = False 2023-01-24T22:07:35.500 INFO:teuthology.orchestra.run.smithi035.stdout: ms inject socket failures = 2500 2023-01-24T22:07:35.500 INFO:teuthology.orchestra.run.smithi035.stdout: ms tcp read timeout = 5 2023-01-24T22:07:35.500 INFO:teuthology.orchestra.run.smithi035.stdout: ms type = async 2023-01-24T22:07:35.500 INFO:teuthology.orchestra.run.smithi035.stdout: osd_max_pg_log_entries = 2 2023-01-24T22:07:35.501 INFO:teuthology.orchestra.run.smithi035.stdout: osd_min_pg_log_entries = 1 2023-01-24T22:07:35.501 INFO:teuthology.orchestra.run.smithi035.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-24T22:07:35.501 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pg_log_trim_min = 0 2023-01-24T22:07:35.501 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pool_default_min_size = 2 2023-01-24T22:07:35.501 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pool_default_size = 3 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout: fsid = 9ce212a6-06d7-42fa-957c-35b0162a9d82 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout:[osd] 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout: osd journal size = 100 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.502 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub load threshold = 5.0 2023-01-24T22:07:35.503 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub max interval = 120 2023-01-24T22:07:35.503 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.503 INFO:teuthology.orchestra.run.smithi035.stdout: osd recover clone overlap = true 2023-01-24T22:07:35.503 INFO:teuthology.orchestra.run.smithi035.stdout: osd recovery max chunk = 1048576 2023-01-24T22:07:35.504 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.504 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug shutdown = true 2023-01-24T22:07:35.504 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug op order = true 2023-01-24T22:07:35.504 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug verify stray on activate = true 2023-01-24T22:07:35.504 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.505 INFO:teuthology.orchestra.run.smithi035.stdout: osd open classes on start = true 2023-01-24T22:07:35.505 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug pg log writeout = true 2023-01-24T22:07:35.505 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.505 INFO:teuthology.orchestra.run.smithi035.stdout: osd deep scrub update digest min age = 30 2023-01-24T22:07:35.505 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: osd map max advance = 10 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: journal zero on create = true 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: filestore ondisk finisher threads = 3 2023-01-24T22:07:35.506 INFO:teuthology.orchestra.run.smithi035.stdout: filestore apply finisher threads = 3 2023-01-24T22:07:35.507 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.507 INFO:teuthology.orchestra.run.smithi035.stdout: bdev debug aio = true 2023-01-24T22:07:35.507 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug misdirected ops = true 2023-01-24T22:07:35.507 INFO:teuthology.orchestra.run.smithi035.stdout: debug monc = 20 2023-01-24T22:07:35.507 INFO:teuthology.orchestra.run.smithi035.stdout: bdev async discard = True 2023-01-24T22:07:35.508 INFO:teuthology.orchestra.run.smithi035.stdout: bdev enable discard = True 2023-01-24T22:07:35.508 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore allocator = bitmap 2023-01-24T22:07:35.508 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore block size = 96636764160 2023-01-24T22:07:35.508 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore fsck on mount = True 2023-01-24T22:07:35.508 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore zero block detection = True 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluefs = 20 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluestore = 20 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: debug osd = 20 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: debug rocksdb = 10 2023-01-24T22:07:35.509 INFO:teuthology.orchestra.run.smithi035.stdout: filestore queue throttle high multiple = 2 2023-01-24T22:07:35.510 INFO:teuthology.orchestra.run.smithi035.stdout: filestore queue throttle max multiple = 10 2023-01-24T22:07:35.510 INFO:teuthology.orchestra.run.smithi035.stdout: journal throttle high multiple = 2 2023-01-24T22:07:35.510 INFO:teuthology.orchestra.run.smithi035.stdout: journal throttle max multiple = 10 2023-01-24T22:07:35.510 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T22:07:35.510 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd full ratio = 0.9 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd nearfull ratio = 0.8 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug verify cached snaps = True 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug verify missing on start = True 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: osd failsafe full ratio = 0.95 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: osd max backfills = 9 2023-01-24T22:07:35.511 INFO:teuthology.orchestra.run.smithi035.stdout: osd max markdown count = 1000 2023-01-24T22:07:35.512 INFO:teuthology.orchestra.run.smithi035.stdout: osd mclock override recovery settings = True 2023-01-24T22:07:35.512 INFO:teuthology.orchestra.run.smithi035.stdout: osd objectstore = bluestore 2023-01-24T22:07:35.512 INFO:teuthology.orchestra.run.smithi035.stdout: osd op queue = debug_random 2023-01-24T22:07:35.512 INFO:teuthology.orchestra.run.smithi035.stdout: osd op queue cut off = debug_random 2023-01-24T22:07:35.512 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub min interval = 60 2023-01-24T22:07:35.513 INFO:teuthology.orchestra.run.smithi035.stdout: osd shutdown pgref assert = True 2023-01-24T22:07:35.513 INFO:teuthology.orchestra.run.smithi035.stdout: osd_max_pg_log_entries = 3000 2023-01-24T22:07:35.513 INFO:teuthology.orchestra.run.smithi035.stdout: osd_min_pg_log_entries = 3000 2023-01-24T22:07:35.513 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.513 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr] 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgr = 20 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2023-01-24T22:07:35.514 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2023-01-24T22:07:35.515 INFO:teuthology.orchestra.run.smithi035.stdout: mgr/telemetry/nag = false 2023-01-24T22:07:35.515 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.515 INFO:teuthology.orchestra.run.smithi035.stdout:[mon] 2023-01-24T22:07:35.515 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2023-01-24T22:07:35.515 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2023-01-24T22:07:35.516 INFO:teuthology.orchestra.run.smithi035.stdout: debug paxos = 20 2023-01-24T22:07:35.516 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2023-01-24T22:07:35.516 INFO:teuthology.orchestra.run.smithi035.stdout: mon data avail warn = 5 2023-01-24T22:07:35.516 INFO:teuthology.orchestra.run.smithi035.stdout: mon mgr mkfs grace = 240 2023-01-24T22:07:35.516 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2023-01-24T22:07:35.517 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd reporter subtree level = osd 2023-01-24T22:07:35.517 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd prime pg temp = true 2023-01-24T22:07:35.517 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2023-01-24T22:07:35.517 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.517 INFO:teuthology.orchestra.run.smithi035.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T22:07:35.518 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T22:07:35.519 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.519 INFO:teuthology.orchestra.run.smithi035.stdout: # 1m isn't quite enough 2023-01-24T22:07:35.519 INFO:teuthology.orchestra.run.smithi035.stdout: mon_down_mkfs_grace = 2m 2023-01-24T22:07:35.519 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.520 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_filestore_osds = false 2023-01-24T22:07:35.520 INFO:teuthology.orchestra.run.smithi035.stdout: mon scrub interval = 300 2023-01-24T22:07:35.520 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:07:35.520 INFO:teuthology.orchestra.run.smithi035.stdout:[client] 2023-01-24T22:07:35.520 INFO:teuthology.orchestra.run.smithi035.stdout: rgw cache enabled = true 2023-01-24T22:07:35.521 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable ops log = true 2023-01-24T22:07:35.521 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable usage log = true 2023-01-24T22:07:35.521 INFO:teuthology.orchestra.run.smithi035.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T22:07:35.521 INFO:teuthology.orchestra.run.smithi035.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T22:07:35.521 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.a] 2023-01-24T22:07:35.522 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.c] 2023-01-24T22:07:35.522 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.b] 2023-01-24T22:07:35.523 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-01-24T22:07:35.523 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-01-24T22:07:35.524 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T22:07:35.524 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-01-24T22:07:35.524 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.524 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-01-24T22:07:35.525 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.526 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-01-24T22:07:35.526 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-01-24T22:07:35.526 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.526 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T22:07:35.526 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-01-24T22:07:35.527 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.528 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-01-24T22:07:35.528 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-01-24T22:07:35.528 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-01-24T22:07:35.528 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-01-24T22:07:35.528 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-01-24T22:07:35.529 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-01-24T22:07:35.529 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-01-24T22:07:35.529 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T22:07:35.529 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T22:07:35.529 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.530 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-01-24T22:07:35.531 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.531 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-01-24T22:07:35.531 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-01-24T22:07:35.531 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-01-24T22:07:35.531 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "[v1:172.21.15.35:6789],[v1:172.21.15.35:6790],[v1:172.21.15.72:6789]" 2023-01-24T22:07:35.532 INFO:teuthology.orchestra.run.smithi072.stdout: mon client directed command retry = 5 2023-01-24T22:07:35.532 INFO:teuthology.orchestra.run.smithi072.stdout: mon election default strategy = 3 2023-01-24T22:07:35.532 INFO:teuthology.orchestra.run.smithi072.stdout: ms bind msgr2 = False 2023-01-24T22:07:35.532 INFO:teuthology.orchestra.run.smithi072.stdout: ms inject socket failures = 2500 2023-01-24T22:07:35.532 INFO:teuthology.orchestra.run.smithi072.stdout: ms tcp read timeout = 5 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: ms type = async 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: osd_max_pg_log_entries = 2 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: osd_min_pg_log_entries = 1 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pg_log_trim_min = 0 2023-01-24T22:07:35.533 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_min_size = 2 2023-01-24T22:07:35.534 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_size = 3 2023-01-24T22:07:35.534 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = 9ce212a6-06d7-42fa-957c-35b0162a9d82 2023-01-24T22:07:35.534 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.534 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-01-24T22:07:35.534 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-01-24T22:07:35.535 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.535 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-01-24T22:07:35.535 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 120 2023-01-24T22:07:35.535 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2023-01-24T22:07:35.536 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-01-24T22:07:35.537 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.537 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-01-24T22:07:35.537 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-01-24T22:07:35.537 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.537 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.538 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-01-24T22:07:35.539 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-01-24T22:07:35.539 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.539 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-01-24T22:07:35.539 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-01-24T22:07:35.539 INFO:teuthology.orchestra.run.smithi072.stdout: debug monc = 20 2023-01-24T22:07:35.540 INFO:teuthology.orchestra.run.smithi072.stdout: bdev async discard = True 2023-01-24T22:07:35.540 INFO:teuthology.orchestra.run.smithi072.stdout: bdev enable discard = True 2023-01-24T22:07:35.540 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore allocator = bitmap 2023-01-24T22:07:35.540 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-01-24T22:07:35.540 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore zero block detection = True 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 20 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 20 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-01-24T22:07:35.541 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 10 2023-01-24T22:07:35.542 INFO:teuthology.orchestra.run.smithi072.stdout: filestore queue throttle high multiple = 2 2023-01-24T22:07:35.542 INFO:teuthology.orchestra.run.smithi072.stdout: filestore queue throttle max multiple = 10 2023-01-24T22:07:35.542 INFO:teuthology.orchestra.run.smithi072.stdout: journal throttle high multiple = 2 2023-01-24T22:07:35.542 INFO:teuthology.orchestra.run.smithi072.stdout: journal throttle max multiple = 10 2023-01-24T22:07:35.542 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify cached snaps = True 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify missing on start = True 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-01-24T22:07:35.543 INFO:teuthology.orchestra.run.smithi072.stdout: osd max backfills = 9 2023-01-24T22:07:35.544 INFO:teuthology.orchestra.run.smithi072.stdout: osd max markdown count = 1000 2023-01-24T22:07:35.544 INFO:teuthology.orchestra.run.smithi072.stdout: osd mclock override recovery settings = True 2023-01-24T22:07:35.544 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-01-24T22:07:35.544 INFO:teuthology.orchestra.run.smithi072.stdout: osd op queue = debug_random 2023-01-24T22:07:35.544 INFO:teuthology.orchestra.run.smithi072.stdout: osd op queue cut off = debug_random 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub min interval = 60 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout: osd shutdown pgref assert = True 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout: osd_max_pg_log_entries = 3000 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout: osd_min_pg_log_entries = 3000 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.545 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-01-24T22:07:35.546 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-01-24T22:07:35.546 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-01-24T22:07:35.546 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-01-24T22:07:35.546 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-01-24T22:07:35.546 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-01-24T22:07:35.547 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-01-24T22:07:35.548 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-01-24T22:07:35.548 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-01-24T22:07:35.548 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-01-24T22:07:35.548 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-01-24T22:07:35.548 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T22:07:35.549 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T22:07:35.550 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T22:07:35.550 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.550 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T22:07:35.550 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T22:07:35.550 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2023-01-24T22:07:35.551 INFO:teuthology.orchestra.run.smithi072.stdout: mon scrub interval = 300 2023-01-24T22:07:35.552 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-01-24T22:07:35.552 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-01-24T22:07:35.552 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-01-24T22:07:35.552 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-01-24T22:07:35.552 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-01-24T22:07:35.553 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T22:07:35.553 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T22:07:35.553 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-01-24T22:07:35.553 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-01-24T22:07:35.553 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-01-24T22:07:35.554 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T22:07:35.554 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T22:07:35.614 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T22:07:35.614 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:35.615 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T22:07:35.631 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:35.631 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T22:07:35.687 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T22:07:35.687 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:35.688 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T22:07:35.688 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T22:07:35.762 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:07:35.762 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:35.817 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T22:07:35.817 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:35.817 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T22:07:35.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T22:07:35.853 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:07:35.853 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:35.907 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T22:07:35.908 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T22:07:35.909 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T22:07:35.956 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T22:07:35.959 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T22:07:36.022 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T22:07:36.025 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T22:07:36.025 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T22:07:36.026 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T22:07:36.064 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T22:07:36.076 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T22:07:36.113 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T22:07:36.125 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T22:07:36.126 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T22:07:36.126 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T22:07:36.151 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:36.152 INFO:tasks.ceph:role: osd.0 2023-01-24T22:07:36.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T22:07:36.153 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T22:07:36.223 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:36.224 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:36.224 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:36.224 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 2023-01-24T22:07:36.224 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:36.225 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:36.225 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:36.225 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:36.225 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:36.225 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:36.605 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2023-01-24T22:07:36.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2023-01-24T22:07:36.607 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T22:07:36.711 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T22:07:36.741 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T22:07:36.807 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:36.807 INFO:tasks.ceph:role: osd.1 2023-01-24T22:07:36.808 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T22:07:36.808 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T22:07:36.880 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:36.881 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:36.881 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:36.881 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 2023-01-24T22:07:36.882 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:36.882 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:36.883 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:36.883 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:36.883 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:36.884 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:37.266 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2023-01-24T22:07:37.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2023-01-24T22:07:37.268 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T22:07:37.396 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T22:07:37.425 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T22:07:37.490 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:37.491 INFO:tasks.ceph:role: osd.2 2023-01-24T22:07:37.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T22:07:37.492 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T22:07:37.563 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:37.563 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:37.563 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:37.563 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 2023-01-24T22:07:37.564 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:37.564 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:37.564 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:37.564 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:37.564 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:37.565 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:37.946 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2023-01-24T22:07:37.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2023-01-24T22:07:37.948 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T22:07:38.042 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T22:07:38.072 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T22:07:38.138 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:38.139 INFO:tasks.ceph:role: osd.3 2023-01-24T22:07:38.139 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com 2023-01-24T22:07:38.139 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T22:07:38.210 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:38.210 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:38.210 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:38.211 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 2023-01-24T22:07:38.211 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:38.211 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:38.211 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:38.211 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:38.212 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:38.212 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:38.597 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2023-01-24T22:07:38.599 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2023-01-24T22:07:38.600 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T22:07:38.705 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T22:07:38.736 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:38.820 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.818+0000 7f5614a68540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T22:07:38.821 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.818+0000 7f5614a68540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T22:07:38.822 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.818+0000 7f5614a68540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T22:07:38.822 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.818+0000 7f5614a68540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T22:07:38.822 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.818+0000 7f5614a68540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T22:07:38.823 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:38.820+0000 7f5614a68540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T22:07:41.682 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T22:07:41.713 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:41.801 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T22:07:41.801 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T22:07:41.801 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T22:07:41.801 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T22:07:41.802 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T22:07:41.802 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:41.798+0000 7f13d693b540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T22:07:44.653 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T22:07:44.681 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:44.768 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T22:07:44.768 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T22:07:44.768 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T22:07:44.769 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T22:07:44.769 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T22:07:44.769 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:44.765+0000 7fe8d73bb540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T22:07:47.623 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T22:07:47.652 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:47.738 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.736+0000 7fd38b580540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T22:07:47.739 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.737+0000 7fd38b580540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T22:07:47.739 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.737+0000 7fd38b580540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T22:07:47.739 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.737+0000 7fd38b580540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T22:07:47.739 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.737+0000 7fd38b580540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T22:07:47.740 INFO:teuthology.orchestra.run.smithi035.stderr:2023-01-24T22:07:47.737+0000 7fd38b580540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T22:07:50.593 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T22:07:50.622 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T22:07:50.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:50.651 INFO:tasks.ceph:role: osd.4 2023-01-24T22:07:50.651 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T22:07:50.652 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T22:07:50.727 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:50.727 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:50.728 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:50.728 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-01-24T22:07:50.728 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:50.728 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:50.729 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:50.729 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:50.729 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:50.729 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:51.096 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-01-24T22:07:51.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-01-24T22:07:51.099 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T22:07:51.212 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T22:07:51.244 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T22:07:51.310 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:51.311 INFO:tasks.ceph:role: osd.5 2023-01-24T22:07:51.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T22:07:51.312 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T22:07:51.383 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:51.383 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:51.384 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:51.384 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-01-24T22:07:51.384 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:51.385 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:51.385 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:51.385 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:51.386 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:51.386 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:51.749 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-01-24T22:07:51.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-01-24T22:07:51.751 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T22:07:51.839 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T22:07:51.869 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T22:07:51.933 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:51.934 INFO:tasks.ceph:role: osd.6 2023-01-24T22:07:51.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T22:07:51.934 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T22:07:52.005 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:52.005 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:52.005 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:52.005 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-01-24T22:07:52.006 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:52.006 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:52.006 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:52.006 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:52.006 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:52.007 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:52.374 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-01-24T22:07:52.376 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-01-24T22:07:52.376 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T22:07:52.468 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T22:07:52.495 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T22:07:52.559 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T22:07:52.559 INFO:tasks.ceph:role: osd.7 2023-01-24T22:07:52.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-01-24T22:07:52.560 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T22:07:52.629 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T22:07:52.630 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T22:07:52.630 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T22:07:52.630 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-01-24T22:07:52.630 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T22:07:52.630 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-01-24T22:07:52.631 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T22:07:52.631 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T22:07:52.631 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T22:07:52.631 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T22:07:53.002 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-01-24T22:07:53.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-01-24T22:07:53.119 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T22:07:53.248 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T22:07:53.278 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:53.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.357+0000 7f2f2b23d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T22:07:53.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.359+0000 7f2f2b23d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T22:07:53.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.360+0000 7f2f2b23d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T22:07:53.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.360+0000 7f2f2b23d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T22:07:53.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.360+0000 7f2f2b23d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T22:07:53.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:53.367+0000 7f2f2b23d540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T22:07:56.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T22:07:56.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:56.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.370+0000 7f184aa40540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T22:07:56.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.371+0000 7f184aa40540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T22:07:56.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.371+0000 7f184aa40540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T22:07:56.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.371+0000 7f184aa40540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T22:07:56.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.371+0000 7f184aa40540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T22:07:56.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:56.371+0000 7f184aa40540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T22:07:59.236 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T22:07:59.263 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:07:59.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T22:07:59.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T22:07:59.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T22:07:59.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T22:07:59.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T22:07:59.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:07:59.345+0000 7f18ab35b540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T22:08:02.198 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T22:08:02.266 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:08:02.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.350+0000 7f5e87183540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T22:08:02.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.350+0000 7f5e87183540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T22:08:02.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.350+0000 7f5e87183540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T22:08:02.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.350+0000 7f5e87183540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T22:08:02.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.350+0000 7f5e87183540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T22:08:02.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-01-24T22:08:02.351+0000 7f5e87183540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T22:08:05.205 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T22:08:05.231 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T22:08:05.232 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.232 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T22:08:05.263 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.263 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T22:08:05.331 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.332 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T22:08:05.398 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.399 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T22:08:05.465 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.466 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T22:08:05.533 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.533 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T22:08:05.561 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T22:08:05.626 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T22:08:05.692 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.692 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T22:08:05.756 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.757 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T22:08:05.820 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:05.820 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T22:08:05.838 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:05.839 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T22:08:05.874 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T22:08:05.875 DEBUG:teuthology.orchestra.run.smithi035:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T22:08:05.880 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T22:08:05.917 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.y] 2023-01-24T22:08:05.918 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCnVtBjDScBORAA8J8cfEh/UIHpvMVVmfHIbA== 2023-01-24T22:08:05.918 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.0] 2023-01-24T22:08:05.918 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCqVtBjK9fiMBAAkKjsB0IELHCeFxYEKhcCyw== 2023-01-24T22:08:05.918 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.1] 2023-01-24T22:08:05.918 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCtVtBj7ZquLxAApMryR2MA09EgGDU49zrxnQ== 2023-01-24T22:08:05.919 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.2] 2023-01-24T22:08:05.919 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCwVtBjura4LRAAPnKwHybMYCMXpuhQZGu+dg== 2023-01-24T22:08:05.919 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.3] 2023-01-24T22:08:05.919 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCzVtBjlp36KxAAcsdvunj9x2MPohT/nAU3eQ== 2023-01-24T22:08:05.920 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.x] 2023-01-24T22:08:05.920 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCoVtBjqixOARAAf5yaHnyoBdBwjl0k83N1AQ== 2023-01-24T22:08:05.920 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.4] 2023-01-24T22:08:05.921 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC5VtBjkDpkFRAABg0oN0L1IJ6SHBN8NglAhw== 2023-01-24T22:08:05.921 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.5] 2023-01-24T22:08:05.921 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC8VtBj9SYtFhAAHRsk8NKoNt077N7iQRCnXg== 2023-01-24T22:08:05.921 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.6] 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC/VtBjzw2jFBAAuDTSA+mD9f1YfHaLYhh3Ng== 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.7] 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDCVtBjidTxFBAAvEvz0qjizIaS4dik5sz94g== 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout:[client.0] 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCoVtBj4FvNAxAAisqg4RXvPd81m7qbozBMyQ== 2023-01-24T22:08:05.922 INFO:teuthology.orchestra.run.smithi035.stdout:[client.1] 2023-01-24T22:08:05.923 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQCoVtBj+oW9BhAAsruPdZsLq4dpFzUnro3VyA== 2023-01-24T22:08:05.936 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-01-24T22:08:05.937 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCnVtBjDScBORAA8J8cfEh/UIHpvMVVmfHIbA== 2023-01-24T22:08:05.937 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-01-24T22:08:05.937 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCqVtBjK9fiMBAAkKjsB0IELHCeFxYEKhcCyw== 2023-01-24T22:08:05.938 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-01-24T22:08:05.938 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCtVtBj7ZquLxAApMryR2MA09EgGDU49zrxnQ== 2023-01-24T22:08:05.938 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-01-24T22:08:05.939 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCwVtBjura4LRAAPnKwHybMYCMXpuhQZGu+dg== 2023-01-24T22:08:05.939 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-01-24T22:08:05.939 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCzVtBjlp36KxAAcsdvunj9x2MPohT/nAU3eQ== 2023-01-24T22:08:05.940 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-01-24T22:08:05.940 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCoVtBjqixOARAAf5yaHnyoBdBwjl0k83N1AQ== 2023-01-24T22:08:05.940 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-01-24T22:08:05.941 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQC5VtBjkDpkFRAABg0oN0L1IJ6SHBN8NglAhw== 2023-01-24T22:08:05.941 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-01-24T22:08:05.941 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQC8VtBj9SYtFhAAHRsk8NKoNt077N7iQRCnXg== 2023-01-24T22:08:05.941 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-01-24T22:08:05.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQC/VtBjzw2jFBAAuDTSA+mD9f1YfHaLYhh3Ng== 2023-01-24T22:08:05.942 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-01-24T22:08:05.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDCVtBjidTxFBAAvEvz0qjizIaS4dik5sz94g== 2023-01-24T22:08:05.943 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-01-24T22:08:05.943 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCoVtBj4FvNAxAAisqg4RXvPd81m7qbozBMyQ== 2023-01-24T22:08:05.943 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2023-01-24T22:08:05.944 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCoVtBj+oW9BhAAsruPdZsLq4dpFzUnro3VyA== 2023-01-24T22:08:05.945 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T22:08:05.967 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T22:08:06.031 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.033 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.082 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.085 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.131 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.133 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.180 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.183 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.231 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T22:08:06.233 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T22:08:06.279 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.282 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.329 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.331 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.377 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.379 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.425 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.428 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T22:08:06.473 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T22:08:06.476 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T22:08:06.524 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T22:08:06.527 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T22:08:06.577 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T22:08:06.577 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T22:08:06.602 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T22:08:06.694 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T22:08:06.719 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T22:08:06.783 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T22:08:06.876 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T22:08:06.901 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T22:08:06.924 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T22:08:07.014 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T22:08:07.037 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:08:07.040 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T22:08:07.091 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T22:08:07.092 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T22:08:07.092 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T22:08:07.095 INFO:tasks.ceph.mon.a:Started 2023-01-24T22:08:07.095 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T22:08:07.095 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T22:08:07.097 INFO:tasks.ceph.mon.c:Started 2023-01-24T22:08:07.097 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T22:08:07.098 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T22:08:07.133 INFO:tasks.ceph.mon.b:Started 2023-01-24T22:08:07.134 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T22:08:07.134 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T22:08:07.136 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T22:08:07.138 INFO:tasks.ceph.mgr.y:Started 2023-01-24T22:08:07.139 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T22:08:07.139 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T22:08:07.141 INFO:tasks.ceph.mgr.x:Started 2023-01-24T22:08:07.141 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T22:08:07.141 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T22:08:13.532 INFO:teuthology.orchestra.run.smithi035.stderr:adjusted tunables profile to default 2023-01-24T22:08:13.543 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T22:08:13.544 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:13.544 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T22:08:13.569 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:13.570 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T22:08:13.633 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:13.634 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T22:08:13.697 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2023-01-24T22:08:13.698 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T22:08:13.762 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:13.763 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T22:08:13.786 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:13.786 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T22:08:13.850 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:13.850 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T22:08:13.912 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-01-24T22:08:13.913 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T22:08:13.975 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 9a8a7db9-2c26-4445-b328-5a965fbc9cf9 0 2023-01-24T22:08:14.222 INFO:teuthology.orchestra.run.smithi072.stdout:0 2023-01-24T22:08:14.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 361be473-2862-4c57-b199-4a3f3a713e00 1 2023-01-24T22:08:14.439 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-01-24T22:08:14.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new fde90b4f-fe4b-4d22-ba26-14f9cc411262 2 2023-01-24T22:08:14.596 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.594+0000 7f401f0e0180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T22:08:14.597 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.594+0000 7f401f0e0180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T22:08:14.598 INFO:tasks.ceph.mgr.x.smithi072.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T22:08:14.598 INFO:tasks.ceph.mgr.x.smithi072.stderr: from .module import Module, StandbyModule 2023-01-24T22:08:14.598 INFO:tasks.ceph.mgr.x.smithi072.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T22:08:14.599 INFO:tasks.ceph.mgr.x.smithi072.stderr: from packaging.version import Version 2023-01-24T22:08:14.599 INFO:tasks.ceph.mgr.x.smithi072.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T22:08:14.599 INFO:tasks.ceph.mgr.x.smithi072.stderr: 2023-01-24T22:08:14.600 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.595+0000 7f401f0e0180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T22:08:14.600 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.595+0000 7f401f0e0180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T22:08:14.644 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.643+0000 7f3621ab8180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T22:08:14.645 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.643+0000 7f3621ab8180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T22:08:14.645 INFO:tasks.ceph.mgr.y.smithi035.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T22:08:14.645 INFO:tasks.ceph.mgr.y.smithi035.stderr: from .module import Module, StandbyModule 2023-01-24T22:08:14.645 INFO:tasks.ceph.mgr.y.smithi035.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T22:08:14.646 INFO:tasks.ceph.mgr.y.smithi035.stderr: from packaging.version import Version 2023-01-24T22:08:14.646 INFO:tasks.ceph.mgr.y.smithi035.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T22:08:14.646 INFO:tasks.ceph.mgr.y.smithi035.stderr: 2023-01-24T22:08:14.646 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.643+0000 7f3621ab8180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T22:08:14.646 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.643+0000 7f3621ab8180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T22:08:14.658 INFO:teuthology.orchestra.run.smithi072.stdout:2 2023-01-24T22:08:14.670 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new bef50b21-75dd-4460-89f9-ce28580655ab 3 2023-01-24T22:08:14.732 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.730+0000 7f401f0e0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T22:08:14.781 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.780+0000 7f3621ab8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T22:08:14.810 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.808+0000 7f401f0e0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T22:08:14.860 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:14.858+0000 7f3621ab8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T22:08:14.877 INFO:teuthology.orchestra.run.smithi072.stdout:3 2023-01-24T22:08:14.890 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 3fe3785d-179e-4164-8914-8d7f0700661d 4 2023-01-24T22:08:14.998 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:14.996+0000 7f401f0e0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T22:08:15.053 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:15.050+0000 7f3621ab8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T22:08:15.096 INFO:teuthology.orchestra.run.smithi072.stdout:4 2023-01-24T22:08:15.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new d32d4dbc-59f9-41e1-b581-c539f53482c2 5 2023-01-24T22:08:15.174 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:15.172+0000 7f401f0e0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T22:08:15.229 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:15.228+0000 7f3621ab8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T22:08:15.254 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:15.252+0000 7f401f0e0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T22:08:15.309 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:15.307+0000 7f3621ab8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T22:08:15.316 INFO:teuthology.orchestra.run.smithi072.stdout:5 2023-01-24T22:08:15.329 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 2ca4febc-902c-419b-b1dc-a56ef94a80fc 6 2023-01-24T22:08:15.393 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:15.391+0000 7f401f0e0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T22:08:15.450 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:15.449+0000 7f3621ab8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T22:08:15.491 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:15.490+0000 7f401f0e0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T22:08:15.535 INFO:teuthology.orchestra.run.smithi072.stdout:6 2023-01-24T22:08:15.548 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new c53abbcf-8848-4514-8f73-151f728381ff 7 2023-01-24T22:08:15.549 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:15.547+0000 7f3621ab8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T22:08:15.754 INFO:teuthology.orchestra.run.smithi072.stdout:7 2023-01-24T22:08:15.767 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T22:08:15.767 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T22:08:15.770 INFO:tasks.ceph.osd.0:Started 2023-01-24T22:08:15.771 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T22:08:15.771 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T22:08:15.773 INFO:tasks.ceph.osd.1:Started 2023-01-24T22:08:15.773 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T22:08:15.774 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T22:08:15.775 INFO:tasks.ceph.osd.2:Started 2023-01-24T22:08:15.776 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T22:08:15.776 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T22:08:15.778 INFO:tasks.ceph.osd.3:Started 2023-01-24T22:08:15.779 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T22:08:15.779 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T22:08:15.781 INFO:tasks.ceph.osd.4:Started 2023-01-24T22:08:15.782 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T22:08:15.782 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T22:08:15.784 INFO:tasks.ceph.osd.5:Started 2023-01-24T22:08:15.785 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T22:08:15.785 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T22:08:15.787 INFO:tasks.ceph.osd.6:Started 2023-01-24T22:08:15.787 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T22:08:15.787 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T22:08:15.790 INFO:tasks.ceph.osd.7:Started 2023-01-24T22:08:15.791 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T22:08:16.023 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:16.023 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":10,"fsid":"9ce212a6-06d7-42fa-957c-35b0162a9d82","created":"2023-01-24T22:08:13.408713+0000","modified":"2023-01-24T22:08:15.749774+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T22:08:15.749774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"9a8a7db9-2c26-4445-b328-5a965fbc9cf9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"361be473-2862-4c57-b199-4a3f3a713e00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fde90b4f-fe4b-4d22-ba26-14f9cc411262","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bef50b21-75dd-4460-89f9-ce28580655ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fe3785d-179e-4164-8914-8d7f0700661d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d32d4dbc-59f9-41e1-b581-c539f53482c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2ca4febc-902c-419b-b1dc-a56ef94a80fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c53abbcf-8848-4514-8f73-151f728381ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T22:08:16.036 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T22:08:16.037 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T22:08:16.037 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.034+0000 7f401f0e0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T22:08:16.111 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.109+0000 7f401f0e0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T22:08:16.115 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:16.114+0000 7f3621ab8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T22:08:16.188 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.186+0000 7f401f0e0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T22:08:16.196 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:16.194+0000 7f3621ab8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T22:08:16.275 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:16.274+0000 7f3621ab8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T22:08:16.369 INFO:tasks.ceph.osd.0.smithi035.stderr:2023-01-24T22:08:16.367+0000 7f9ba6b81540 -1 Falling back to public interface 2023-01-24T22:08:16.380 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-01-24T22:08:16.378+0000 7fc98a8e4540 -1 Falling back to public interface 2023-01-24T22:08:16.383 INFO:tasks.ceph.osd.1.smithi035.stderr:2023-01-24T22:08:16.381+0000 7fbfa648a540 -1 Falling back to public interface 2023-01-24T22:08:16.384 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-01-24T22:08:16.382+0000 7fcd84709540 -1 Falling back to public interface 2023-01-24T22:08:16.386 INFO:tasks.ceph.osd.3.smithi035.stderr:2023-01-24T22:08:16.385+0000 7f5ac23c0540 -1 Falling back to public interface 2023-01-24T22:08:16.401 INFO:tasks.ceph.osd.2.smithi035.stderr:2023-01-24T22:08:16.396+0000 7f0dc3913540 -1 Falling back to public interface 2023-01-24T22:08:16.405 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-01-24T22:08:16.403+0000 7f85bb4f5540 -1 Falling back to public interface 2023-01-24T22:08:16.417 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-01-24T22:08:16.415+0000 7fb8265dc540 -1 Falling back to public interface 2023-01-24T22:08:16.749 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.747+0000 7f401f0e0180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T22:08:16.842 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T22:08:16.883 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.882+0000 7f401f0e0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T22:08:16.891 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:16.889+0000 7f3621ab8180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T22:08:16.966 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:16.964+0000 7f401f0e0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T22:08:17.028 INFO:teuthology.misc.health.smithi035.stdout: 2023-01-24T22:08:17.028 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":10,"fsid":"9ce212a6-06d7-42fa-957c-35b0162a9d82","created":"2023-01-24T22:08:13.408713+0000","modified":"2023-01-24T22:08:15.749774+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T22:08:15.749774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"9a8a7db9-2c26-4445-b328-5a965fbc9cf9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"361be473-2862-4c57-b199-4a3f3a713e00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fde90b4f-fe4b-4d22-ba26-14f9cc411262","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bef50b21-75dd-4460-89f9-ce28580655ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fe3785d-179e-4164-8914-8d7f0700661d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d32d4dbc-59f9-41e1-b581-c539f53482c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2ca4febc-902c-419b-b1dc-a56ef94a80fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c53abbcf-8848-4514-8f73-151f728381ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T22:08:17.036 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:17.035+0000 7f3621ab8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T22:08:17.039 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T22:08:17.134 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:17.132+0000 7f3621ab8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T22:08:17.209 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:17.207+0000 7f401f0e0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T22:08:17.287 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:17.285+0000 7f401f0e0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T22:08:17.391 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:17.389+0000 7f3621ab8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T22:08:17.471 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:17.470+0000 7f3621ab8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T22:08:17.852 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:17.850+0000 7f401f0e0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T22:08:18.094 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.092+0000 7f3621ab8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T22:08:18.103 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.101+0000 7f401f0e0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T22:08:18.232 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.230+0000 7f401f0e0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T22:08:18.314 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.312+0000 7f401f0e0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T22:08:18.343 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.341+0000 7f3621ab8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T22:08:18.462 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.460+0000 7f401f0e0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T22:08:18.482 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.480+0000 7f3621ab8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T22:08:18.534 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.532+0000 7f401f0e0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T22:08:18.569 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.567+0000 7f3621ab8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T22:08:18.609 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.607+0000 7f401f0e0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T22:08:18.682 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.681+0000 7f401f0e0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T22:08:18.683 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:18.681+0000 7f401f0e0180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T22:08:18.726 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.724+0000 7f3621ab8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T22:08:18.809 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.808+0000 7f3621ab8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T22:08:18.890 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.889+0000 7f3621ab8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T22:08:18.967 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.966+0000 7f3621ab8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T22:08:18.968 INFO:tasks.ceph.mgr.y.smithi035.stderr:2023-01-24T22:08:18.966+0000 7f3621ab8180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T22:08:19.706 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:19.705+0000 7f3fd85aa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T22:08:19.707 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-01-24T22:08:19.705+0000 7f3fd85aa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T22:08:19.867 INFO:tasks.ceph.osd.0.smithi035.stderr:2023-01-24T22:08:19.865+0000 7f9ba6b81540 -1 osd.0 0 log_to_monitors true 2023-01-24T22:08:19.919 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-01-24T22:08:19.918+0000 7fcd84709540 -1 osd.6 0 log_to_monitors true 2023-01-24T22:08:19.930 INFO:tasks.ceph.osd.1.smithi035.stderr:2023-01-24T22:08:19.929+0000 7fbfa648a540 -1 osd.1 0 log_to_monitors true 2023-01-24T22:08:19.938 INFO:tasks.ceph.osd.3.smithi035.stderr:2023-01-24T22:08:19.936+0000 7f5ac23c0540 -1 osd.3 0 log_to_monitors true 2023-01-24T22:08:19.946 INFO:tasks.ceph.osd.2.smithi035.stderr:2023-01-24T22:08:19.945+0000 7f0dc3913540 -1 osd.2 0 log_to_monitors true 2023-01-24T22:08:19.951 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-01-24T22:08:19.951+0000 7fc98a8e4540 -1 osd.4 0 log_to_monitors true 2023-01-24T22:08:19.973 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-01-24T22:08:19.972+0000 7f85bb4f5540 -1 osd.5 0 log_to_monitors true 2023-01-24T22:08:19.974 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-01-24T22:08:19.972+0000 7fb8265dc540 -1 osd.7 0 log_to_monitors true 2023-01-24T22:08:21.774 INFO:tasks.ceph.osd.2.smithi035.stderr:2023-01-24T22:08:21.772+0000 7f0dba534700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T22:08:21.775 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-01-24T22:08:21.774+0000 7fcd7b32a700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T22:08:21.775 INFO:tasks.ceph.osd.0.smithi035.stderr:2023-01-24T22:08:21.773+0000 7f9b9efa5700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T22:08:21.776 INFO:tasks.ceph.osd.1.smithi035.stderr:2023-01-24T22:08:21.773+0000 7fbf9d0ab700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T22:08:21.776 INFO:tasks.ceph.osd.3.smithi035.stderr:2023-01-24T22:08:21.774+0000 7f5ab8fe1700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T22:08:21.777 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-01-24T22:08:21.776+0000 7fc981505700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T22:08:21.777 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-01-24T22:08:21.776+0000 7f85b2116700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T22:08:21.777 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-01-24T22:08:21.777+0000 7fb81d1fd700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T22:08:21.779 INFO:tasks.ceph.osd.2.smithi035.stderr:2023-01-24T22:08:21.778+0000 7f0db532e700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.779 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-01-24T22:08:21.779+0000 7fcd76124700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.780 INFO:tasks.ceph.osd.1.smithi035.stderr:2023-01-24T22:08:21.778+0000 7fbf97ea5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.780 INFO:tasks.ceph.osd.0.smithi035.stderr:2023-01-24T22:08:21.779+0000 7f9b98d9d700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.781 INFO:tasks.ceph.osd.3.smithi035.stderr:2023-01-24T22:08:21.779+0000 7f5ab3ddb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.782 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-01-24T22:08:21.782+0000 7fc97c2ff700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.783 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-01-24T22:08:21.782+0000 7f85acf10700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:21.783 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-01-24T22:08:21.782+0000 7fb817ff7700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T22:08:23.847 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T22:08:24.167 INFO:teuthology.misc.health.smithi035.stdout: 2023-01-24T22:08:24.168 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":14,"fsid":"9ce212a6-06d7-42fa-957c-35b0162a9d82","created":"2023-01-24T22:08:13.408713+0000","modified":"2023-01-24T22:08:23.777239+0000","last_up_change":"2023-01-24T22:08:22.769335+0000","last_in_change":"2023-01-24T22:08:15.749774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"9a8a7db9-2c26-4445-b328-5a965fbc9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6800","nonce":82448}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6801","nonce":82448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6803","nonce":82448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6802","nonce":82448}]},"public_addr":"172.21.15.35:6800/82448","cluster_addr":"172.21.15.35:6801/82448","heartbeat_back_addr":"172.21.15.35:6803/82448","heartbeat_front_addr":"172.21.15.35:6802/82448","state":["exists","up"]},{"osd":1,"uuid":"361be473-2862-4c57-b199-4a3f3a713e00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6804","nonce":82449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6805","nonce":82449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6807","nonce":82449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6806","nonce":82449}]},"public_addr":"172.21.15.35:6804/82449","cluster_addr":"172.21.15.35:6805/82449","heartbeat_back_addr":"172.21.15.35:6807/82449","heartbeat_front_addr":"172.21.15.35:6806/82449","state":["exists","up"]},{"osd":2,"uuid":"fde90b4f-fe4b-4d22-ba26-14f9cc411262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6812","nonce":82474}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6813","nonce":82474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6815","nonce":82474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6814","nonce":82474}]},"public_addr":"172.21.15.35:6812/82474","cluster_addr":"172.21.15.35:6813/82474","heartbeat_back_addr":"172.21.15.35:6815/82474","heartbeat_front_addr":"172.21.15.35:6814/82474","state":["exists","up"]},{"osd":3,"uuid":"bef50b21-75dd-4460-89f9-ce28580655ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6808","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6809","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6811","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6810","nonce":82464}]},"public_addr":"172.21.15.35:6808/82464","cluster_addr":"172.21.15.35:6809/82464","heartbeat_back_addr":"172.21.15.35:6811/82464","heartbeat_front_addr":"172.21.15.35:6810/82464","state":["exists","up"]},{"osd":4,"uuid":"3fe3785d-179e-4164-8914-8d7f0700661d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":82427}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":82427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":82427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":82427}]},"public_addr":"172.21.15.72:6800/82427","cluster_addr":"172.21.15.72:6801/82427","heartbeat_back_addr":"172.21.15.72:6803/82427","heartbeat_front_addr":"172.21.15.72:6802/82427","state":["exists","up"]},{"osd":5,"uuid":"d32d4dbc-59f9-41e1-b581-c539f53482c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":82457}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":82457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":82457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":82457}]},"public_addr":"172.21.15.72:6808/82457","cluster_addr":"172.21.15.72:6809/82457","heartbeat_back_addr":"172.21.15.72:6811/82457","heartbeat_front_addr":"172.21.15.72:6810/82457","state":["exists","up"]},{"osd":6,"uuid":"2ca4febc-902c-419b-b1dc-a56ef94a80fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":82428}]},"public_addr":"172.21.15.72:6804/82428","cluster_addr":"172.21.15.72:6805/82428","heartbeat_back_addr":"172.21.15.72:6807/82428","heartbeat_front_addr":"172.21.15.72:6806/82428","state":["exists","up"]},{"osd":7,"uuid":"c53abbcf-8848-4514-8f73-151f728381ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6812","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6813","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6815","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6814","nonce":82458}]},"public_addr":"172.21.15.72:6812/82458","cluster_addr":"172.21.15.72:6813/82458","heartbeat_back_addr":"172.21.15.72:6815/82458","heartbeat_front_addr":"172.21.15.72:6814/82458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.860341+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.967727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.996865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.959237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.001652+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.960488+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.965421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.021029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T22:08:24.180 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T22:08:24.181 INFO:tasks.ceph:Creating RBD pool 2023-01-24T22:08:24.181 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T22:08:24.794 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'rbd' created 2023-01-24T22:08:24.805 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T22:08:25.803 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T22:08:25.817 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T22:08:25.818 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T22:08:25.818 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T22:08:25.819 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T22:08:25.819 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T22:08:26.221 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:26.234 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":4,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6816","nonce":81880}]},"active_addr":"172.21.15.72:6816/81880","active_change":"2023-01-24T22:08:18.695123+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1134467636}]},{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":936781393}]},{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2779245931}]}]}} 2023-01-24T22:08:26.238 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T22:08:26.238 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T22:08:26.239 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T22:08:26.564 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:26.565 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":16,"fsid":"9ce212a6-06d7-42fa-957c-35b0162a9d82","created":"2023-01-24T22:08:13.408713+0000","modified":"2023-01-24T22:08:25.787830+0000","last_up_change":"2023-01-24T22:08:22.769335+0000","last_in_change":"2023-01-24T22:08:15.749774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T22:08:24.526459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T22:08:24.742823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"9a8a7db9-2c26-4445-b328-5a965fbc9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6800","nonce":82448}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6801","nonce":82448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6803","nonce":82448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6802","nonce":82448}]},"public_addr":"172.21.15.35:6800/82448","cluster_addr":"172.21.15.35:6801/82448","heartbeat_back_addr":"172.21.15.35:6803/82448","heartbeat_front_addr":"172.21.15.35:6802/82448","state":["exists","up"]},{"osd":1,"uuid":"361be473-2862-4c57-b199-4a3f3a713e00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6804","nonce":82449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6805","nonce":82449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6807","nonce":82449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6806","nonce":82449}]},"public_addr":"172.21.15.35:6804/82449","cluster_addr":"172.21.15.35:6805/82449","heartbeat_back_addr":"172.21.15.35:6807/82449","heartbeat_front_addr":"172.21.15.35:6806/82449","state":["exists","up"]},{"osd":2,"uuid":"fde90b4f-fe4b-4d22-ba26-14f9cc411262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6812","nonce":82474}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6813","nonce":82474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6815","nonce":82474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6814","nonce":82474}]},"public_addr":"172.21.15.35:6812/82474","cluster_addr":"172.21.15.35:6813/82474","heartbeat_back_addr":"172.21.15.35:6815/82474","heartbeat_front_addr":"172.21.15.35:6814/82474","state":["exists","up"]},{"osd":3,"uuid":"bef50b21-75dd-4460-89f9-ce28580655ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6808","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6809","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6811","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6810","nonce":82464}]},"public_addr":"172.21.15.35:6808/82464","cluster_addr":"172.21.15.35:6809/82464","heartbeat_back_addr":"172.21.15.35:6811/82464","heartbeat_front_addr":"172.21.15.35:6810/82464","state":["exists","up"]},{"osd":4,"uuid":"3fe3785d-179e-4164-8914-8d7f0700661d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":82427}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":82427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":82427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":82427}]},"public_addr":"172.21.15.72:6800/82427","cluster_addr":"172.21.15.72:6801/82427","heartbeat_back_addr":"172.21.15.72:6803/82427","heartbeat_front_addr":"172.21.15.72:6802/82427","state":["exists","up"]},{"osd":5,"uuid":"d32d4dbc-59f9-41e1-b581-c539f53482c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":82457}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":82457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":82457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":82457}]},"public_addr":"172.21.15.72:6808/82457","cluster_addr":"172.21.15.72:6809/82457","heartbeat_back_addr":"172.21.15.72:6811/82457","heartbeat_front_addr":"172.21.15.72:6810/82457","state":["exists","up"]},{"osd":6,"uuid":"2ca4febc-902c-419b-b1dc-a56ef94a80fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":82428}]},"public_addr":"172.21.15.72:6804/82428","cluster_addr":"172.21.15.72:6805/82428","heartbeat_back_addr":"172.21.15.72:6807/82428","heartbeat_front_addr":"172.21.15.72:6806/82428","state":["exists","up"]},{"osd":7,"uuid":"c53abbcf-8848-4514-8f73-151f728381ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6812","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6813","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6815","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6814","nonce":82458}]},"public_addr":"172.21.15.72:6812/82458","cluster_addr":"172.21.15.72:6813/82458","heartbeat_back_addr":"172.21.15.72:6815/82458","heartbeat_front_addr":"172.21.15.72:6814/82458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.860341+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.967727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.996865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.959237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.001652+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.960488+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.965421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.021029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T22:08:26.578 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T22:08:26.578 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T22:08:26.932 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:26.932 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":17,"fsid":"9ce212a6-06d7-42fa-957c-35b0162a9d82","created":"2023-01-24T22:08:13.408713+0000","modified":"2023-01-24T22:08:26.795296+0000","last_up_change":"2023-01-24T22:08:22.769335+0000","last_in_change":"2023-01-24T22:08:15.749774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T22:08:24.526459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T22:08:24.742823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"9a8a7db9-2c26-4445-b328-5a965fbc9cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6800","nonce":82448}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6801","nonce":82448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6803","nonce":82448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6802","nonce":82448}]},"public_addr":"172.21.15.35:6800/82448","cluster_addr":"172.21.15.35:6801/82448","heartbeat_back_addr":"172.21.15.35:6803/82448","heartbeat_front_addr":"172.21.15.35:6802/82448","state":["exists","up"]},{"osd":1,"uuid":"361be473-2862-4c57-b199-4a3f3a713e00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6804","nonce":82449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6805","nonce":82449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6807","nonce":82449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6806","nonce":82449}]},"public_addr":"172.21.15.35:6804/82449","cluster_addr":"172.21.15.35:6805/82449","heartbeat_back_addr":"172.21.15.35:6807/82449","heartbeat_front_addr":"172.21.15.35:6806/82449","state":["exists","up"]},{"osd":2,"uuid":"fde90b4f-fe4b-4d22-ba26-14f9cc411262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6812","nonce":82474}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6813","nonce":82474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6815","nonce":82474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6814","nonce":82474}]},"public_addr":"172.21.15.35:6812/82474","cluster_addr":"172.21.15.35:6813/82474","heartbeat_back_addr":"172.21.15.35:6815/82474","heartbeat_front_addr":"172.21.15.35:6814/82474","state":["exists","up"]},{"osd":3,"uuid":"bef50b21-75dd-4460-89f9-ce28580655ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6808","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6809","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6811","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.35:6810","nonce":82464}]},"public_addr":"172.21.15.35:6808/82464","cluster_addr":"172.21.15.35:6809/82464","heartbeat_back_addr":"172.21.15.35:6811/82464","heartbeat_front_addr":"172.21.15.35:6810/82464","state":["exists","up"]},{"osd":4,"uuid":"3fe3785d-179e-4164-8914-8d7f0700661d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":82427}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":82427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":82427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":82427}]},"public_addr":"172.21.15.72:6800/82427","cluster_addr":"172.21.15.72:6801/82427","heartbeat_back_addr":"172.21.15.72:6803/82427","heartbeat_front_addr":"172.21.15.72:6802/82427","state":["exists","up"]},{"osd":5,"uuid":"d32d4dbc-59f9-41e1-b581-c539f53482c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":82457}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":82457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":82457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":82457}]},"public_addr":"172.21.15.72:6808/82457","cluster_addr":"172.21.15.72:6809/82457","heartbeat_back_addr":"172.21.15.72:6811/82457","heartbeat_front_addr":"172.21.15.72:6810/82457","state":["exists","up"]},{"osd":6,"uuid":"2ca4febc-902c-419b-b1dc-a56ef94a80fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":82428}]},"public_addr":"172.21.15.72:6804/82428","cluster_addr":"172.21.15.72:6805/82428","heartbeat_back_addr":"172.21.15.72:6807/82428","heartbeat_front_addr":"172.21.15.72:6806/82428","state":["exists","up"]},{"osd":7,"uuid":"c53abbcf-8848-4514-8f73-151f728381ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6812","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6813","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6815","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6814","nonce":82458}]},"public_addr":"172.21.15.72:6812/82458","cluster_addr":"172.21.15.72:6813/82458","heartbeat_back_addr":"172.21.15.72:6815/82458","heartbeat_front_addr":"172.21.15.72:6814/82458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.860341+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.967727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.996865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.959237+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.001652+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.960488+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:20.965421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T22:08:21.021029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T22:08:26.949 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T22:08:26.950 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T22:08:26.950 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T22:08:26.951 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T22:08:26.951 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T22:08:26.952 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T22:08:26.952 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T22:08:26.953 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T22:08:27.209 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.210 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T22:08:27.219 INFO:teuthology.orchestra.run.smithi035.stdout:55834574852 2023-01-24T22:08:27.220 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T22:08:27.226 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:27.227 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T22:08:27.238 INFO:teuthology.orchestra.run.smithi035.stdout:55834574852 2023-01-24T22:08:27.239 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T22:08:27.242 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:27.242 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T22:08:27.246 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:27.247 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T22:08:27.259 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.260 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T22:08:27.262 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.262 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T22:08:27.849 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:27.850 INFO:teuthology.orchestra.run.smithi035.stdout:55834574849 2023-01-24T22:08:27.863 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.866 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:27.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2023-01-24T22:08:27.872 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2023-01-24T22:08:27.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-01-24T22:08:27.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-24T22:08:27.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-24T22:08:27.910 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:27.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-01-24T22:08:27.949 INFO:teuthology.orchestra.run.smithi035.stdout:55834574849 2023-01-24T22:08:27.958 INFO:teuthology.orchestra.run.smithi035.stdout:55834574849 2023-01-24T22:08:27.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2023-01-24T22:08:27.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.3 2023-01-24T22:08:28.872 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T22:08:28.874 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T22:08:28.888 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T22:08:28.893 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T22:08:28.896 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T22:08:28.926 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T22:08:28.965 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T22:08:28.973 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T22:08:29.459 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:29.471 INFO:teuthology.orchestra.run.smithi035.stdout:55834574852 2023-01-24T22:08:29.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.0 2023-01-24T22:08:29.489 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2023-01-24T22:08:29.494 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.526 INFO:teuthology.orchestra.run.smithi035.stdout:55834574852 2023-01-24T22:08:29.547 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:29.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-01-24T22:08:29.548 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-24T22:08:29.560 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.570 INFO:teuthology.orchestra.run.smithi035.stdout:55834574852 2023-01-24T22:08:29.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.7 2023-01-24T22:08:29.589 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.592 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:29.593 INFO:teuthology.orchestra.run.smithi035.stdout:55834574851 2023-01-24T22:08:29.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.4 2023-01-24T22:08:29.608 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-24T22:08:29.614 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.630 INFO:teuthology.orchestra.run.smithi035.stdout:55834574850 2023-01-24T22:08:29.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.3 2023-01-24T22:08:29.644 DEBUG:teuthology.parallel:result is None 2023-01-24T22:08:29.644 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T22:08:29.645 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T22:08:29.998 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:30.001 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2023-01-24T22:08:30.012 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T22:08:28.700247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44148,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754930572,"statfs":{"total":773094113280,"available":773048905728,"internally_reserved":0,"allocated":2621440,"data_stored":1808728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803109+0000","last_change":"2023-01-24T22:08:25.803109+0000","last_active":"2023-01-24T22:08:25.803109+0000","last_peered":"2023-01-24T22:08:25.803109+0000","last_clean":"2023-01-24T22:08:25.803109+0000","last_became_active":"2023-01-24T22:08:25.802774+0000","last_became_peered":"2023-01-24T22:08:25.802774+0000","last_unstale":"2023-01-24T22:08:25.803109+0000","last_undegraded":"2023-01-24T22:08:25.803109+0000","last_fullsized":"2023-01-24T22:08:25.803109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.802558+0000","last_change":"2023-01-24T22:08:25.802558+0000","last_active":"2023-01-24T22:08:25.802558+0000","last_peered":"2023-01-24T22:08:25.802558+0000","last_clean":"2023-01-24T22:08:25.802558+0000","last_became_active":"2023-01-24T22:08:25.802204+0000","last_became_peered":"2023-01-24T22:08:25.802204+0000","last_unstale":"2023-01-24T22:08:25.802558+0000","last_undegraded":"2023-01-24T22:08:25.802558+0000","last_fullsized":"2023-01-24T22:08:25.802558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.157213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803654+0000","last_change":"2023-01-24T22:08:25.801490+0000","last_active":"2023-01-24T22:08:25.803654+0000","last_peered":"2023-01-24T22:08:25.803654+0000","last_clean":"2023-01-24T22:08:25.803654+0000","last_became_active":"2023-01-24T22:08:25.801046+0000","last_became_peered":"2023-01-24T22:08:25.801046+0000","last_unstale":"2023-01-24T22:08:25.803654+0000","last_undegraded":"2023-01-24T22:08:25.803654+0000","last_fullsized":"2023-01-24T22:08:25.803654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:42.654737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804977+0000","last_change":"2023-01-24T22:08:25.802315+0000","last_active":"2023-01-24T22:08:25.804977+0000","last_peered":"2023-01-24T22:08:25.804977+0000","last_clean":"2023-01-24T22:08:25.804977+0000","last_became_active":"2023-01-24T22:08:25.802003+0000","last_became_peered":"2023-01-24T22:08:25.802003+0000","last_unstale":"2023-01-24T22:08:25.804977+0000","last_undegraded":"2023-01-24T22:08:25.804977+0000","last_fullsized":"2023-01-24T22:08:25.804977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:28.599628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803128+0000","last_change":"2023-01-24T22:08:25.799680+0000","last_active":"2023-01-24T22:08:25.803128+0000","last_peered":"2023-01-24T22:08:25.803128+0000","last_clean":"2023-01-24T22:08:25.803128+0000","last_became_active":"2023-01-24T22:08:25.799419+0000","last_became_peered":"2023-01-24T22:08:25.799419+0000","last_unstale":"2023-01-24T22:08:25.803128+0000","last_undegraded":"2023-01-24T22:08:25.803128+0000","last_fullsized":"2023-01-24T22:08:25.803128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:43.005484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T22:08:26.948706+0000","last_change":"2023-01-24T22:08:25.799400+0000","last_active":"2023-01-24T22:08:26.948706+0000","last_peered":"2023-01-24T22:08:26.948706+0000","last_clean":"2023-01-24T22:08:26.948706+0000","last_became_active":"2023-01-24T22:08:25.799019+0000","last_became_peered":"2023-01-24T22:08:25.799019+0000","last_unstale":"2023-01-24T22:08:26.948706+0000","last_undegraded":"2023-01-24T22:08:26.948706+0000","last_fullsized":"2023-01-24T22:08:26.948706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804798+0000","last_change":"2023-01-24T22:08:25.803755+0000","last_active":"2023-01-24T22:08:25.804798+0000","last_peered":"2023-01-24T22:08:25.804798+0000","last_clean":"2023-01-24T22:08:25.804798+0000","last_became_active":"2023-01-24T22:08:25.803357+0000","last_became_peered":"2023-01-24T22:08:25.803357+0000","last_unstale":"2023-01-24T22:08:25.804798+0000","last_undegraded":"2023-01-24T22:08:25.804798+0000","last_fullsized":"2023-01-24T22:08:25.804798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:25.269566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804048+0000","last_change":"2023-01-24T22:08:25.804048+0000","last_active":"2023-01-24T22:08:25.804048+0000","last_peered":"2023-01-24T22:08:25.804048+0000","last_clean":"2023-01-24T22:08:25.804048+0000","last_became_active":"2023-01-24T22:08:25.803718+0000","last_became_peered":"2023-01-24T22:08:25.803718+0000","last_unstale":"2023-01-24T22:08:25.804048+0000","last_undegraded":"2023-01-24T22:08:25.804048+0000","last_fullsized":"2023-01-24T22:08:25.804048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.933690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.802276+0000","last_change":"2023-01-24T22:08:25.802276+0000","last_active":"2023-01-24T22:08:25.802276+0000","last_peered":"2023-01-24T22:08:25.802276+0000","last_clean":"2023-01-24T22:08:25.802276+0000","last_became_active":"2023-01-24T22:08:25.801955+0000","last_became_peered":"2023-01-24T22:08:25.801955+0000","last_unstale":"2023-01-24T22:08:25.802276+0000","last_undegraded":"2023-01-24T22:08:25.802276+0000","last_fullsized":"2023-01-24T22:08:25.802276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:37.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5119999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4610000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6232,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365608,"statfs":{"total":96636764160,"available":96630382592,"internally_reserved":0,"allocated":610304,"data_stored":508098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":618496,"data_stored":513084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T22:08:30.013 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T22:08:30.327 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:30.328 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2023-01-24T22:08:30.340 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T22:08:28.700247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44148,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":40448,"kb_avail":754930572,"statfs":{"total":773094113280,"available":773048905728,"internally_reserved":0,"allocated":2621440,"data_stored":1808728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41418752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803109+0000","last_change":"2023-01-24T22:08:25.803109+0000","last_active":"2023-01-24T22:08:25.803109+0000","last_peered":"2023-01-24T22:08:25.803109+0000","last_clean":"2023-01-24T22:08:25.803109+0000","last_became_active":"2023-01-24T22:08:25.802774+0000","last_became_peered":"2023-01-24T22:08:25.802774+0000","last_unstale":"2023-01-24T22:08:25.803109+0000","last_undegraded":"2023-01-24T22:08:25.803109+0000","last_fullsized":"2023-01-24T22:08:25.803109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.802558+0000","last_change":"2023-01-24T22:08:25.802558+0000","last_active":"2023-01-24T22:08:25.802558+0000","last_peered":"2023-01-24T22:08:25.802558+0000","last_clean":"2023-01-24T22:08:25.802558+0000","last_became_active":"2023-01-24T22:08:25.802204+0000","last_became_peered":"2023-01-24T22:08:25.802204+0000","last_unstale":"2023-01-24T22:08:25.802558+0000","last_undegraded":"2023-01-24T22:08:25.802558+0000","last_fullsized":"2023-01-24T22:08:25.802558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.157213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803654+0000","last_change":"2023-01-24T22:08:25.801490+0000","last_active":"2023-01-24T22:08:25.803654+0000","last_peered":"2023-01-24T22:08:25.803654+0000","last_clean":"2023-01-24T22:08:25.803654+0000","last_became_active":"2023-01-24T22:08:25.801046+0000","last_became_peered":"2023-01-24T22:08:25.801046+0000","last_unstale":"2023-01-24T22:08:25.803654+0000","last_undegraded":"2023-01-24T22:08:25.803654+0000","last_fullsized":"2023-01-24T22:08:25.803654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:42.654737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804977+0000","last_change":"2023-01-24T22:08:25.802315+0000","last_active":"2023-01-24T22:08:25.804977+0000","last_peered":"2023-01-24T22:08:25.804977+0000","last_clean":"2023-01-24T22:08:25.804977+0000","last_became_active":"2023-01-24T22:08:25.802003+0000","last_became_peered":"2023-01-24T22:08:25.802003+0000","last_unstale":"2023-01-24T22:08:25.804977+0000","last_undegraded":"2023-01-24T22:08:25.804977+0000","last_fullsized":"2023-01-24T22:08:25.804977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:28.599628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803128+0000","last_change":"2023-01-24T22:08:25.799680+0000","last_active":"2023-01-24T22:08:25.803128+0000","last_peered":"2023-01-24T22:08:25.803128+0000","last_clean":"2023-01-24T22:08:25.803128+0000","last_became_active":"2023-01-24T22:08:25.799419+0000","last_became_peered":"2023-01-24T22:08:25.799419+0000","last_unstale":"2023-01-24T22:08:25.803128+0000","last_undegraded":"2023-01-24T22:08:25.803128+0000","last_fullsized":"2023-01-24T22:08:25.803128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:43.005484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T22:08:26.948706+0000","last_change":"2023-01-24T22:08:25.799400+0000","last_active":"2023-01-24T22:08:26.948706+0000","last_peered":"2023-01-24T22:08:26.948706+0000","last_clean":"2023-01-24T22:08:26.948706+0000","last_became_active":"2023-01-24T22:08:25.799019+0000","last_became_peered":"2023-01-24T22:08:25.799019+0000","last_unstale":"2023-01-24T22:08:26.948706+0000","last_undegraded":"2023-01-24T22:08:26.948706+0000","last_fullsized":"2023-01-24T22:08:26.948706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804798+0000","last_change":"2023-01-24T22:08:25.803755+0000","last_active":"2023-01-24T22:08:25.804798+0000","last_peered":"2023-01-24T22:08:25.804798+0000","last_clean":"2023-01-24T22:08:25.804798+0000","last_became_active":"2023-01-24T22:08:25.803357+0000","last_became_peered":"2023-01-24T22:08:25.803357+0000","last_unstale":"2023-01-24T22:08:25.804798+0000","last_undegraded":"2023-01-24T22:08:25.804798+0000","last_fullsized":"2023-01-24T22:08:25.804798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:25.269566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804048+0000","last_change":"2023-01-24T22:08:25.804048+0000","last_active":"2023-01-24T22:08:25.804048+0000","last_peered":"2023-01-24T22:08:25.804048+0000","last_clean":"2023-01-24T22:08:25.804048+0000","last_became_active":"2023-01-24T22:08:25.803718+0000","last_became_peered":"2023-01-24T22:08:25.803718+0000","last_unstale":"2023-01-24T22:08:25.804048+0000","last_undegraded":"2023-01-24T22:08:25.804048+0000","last_fullsized":"2023-01-24T22:08:25.804048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.933690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.802276+0000","last_change":"2023-01-24T22:08:25.802276+0000","last_active":"2023-01-24T22:08:25.802276+0000","last_peered":"2023-01-24T22:08:25.802276+0000","last_clean":"2023-01-24T22:08:25.802276+0000","last_became_active":"2023-01-24T22:08:25.801955+0000","last_became_peered":"2023-01-24T22:08:25.801955+0000","last_unstale":"2023-01-24T22:08:25.802276+0000","last_undegraded":"2023-01-24T22:08:25.802276+0000","last_fullsized":"2023-01-24T22:08:25.802276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:37.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5119999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4610000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6232,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365608,"statfs":{"total":96636764160,"available":96630382592,"internally_reserved":0,"allocated":610304,"data_stored":508098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":618496,"data_stored":513084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T22:08:30.341 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T22:08:30.342 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T22:08:30.342 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T22:08:30.342 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T22:08:30.732 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:30.732 INFO:teuthology.orchestra.run.smithi035.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T22:08:30.745 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T22:08:30.746 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-24T22:08:30.756 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-24T22:08:30.756 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T22:08:30.757 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-24T22:08:30.757 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-24T22:08:31.094 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-24T22:08:31.094 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T22:08:31.095 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T22:08:31.406 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:31.408 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2023-01-24T22:08:31.419 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T22:08:30.700686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44748,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":40640,"kb_avail":754929972,"statfs":{"total":773094113280,"available":773048291328,"internally_reserved":0,"allocated":2777088,"data_stored":1951552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41615360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.805162+0000","last_change":"2023-01-24T22:08:25.803109+0000","last_active":"2023-01-24T22:08:25.805162+0000","last_peered":"2023-01-24T22:08:25.805162+0000","last_clean":"2023-01-24T22:08:25.805162+0000","last_became_active":"2023-01-24T22:08:25.802774+0000","last_became_peered":"2023-01-24T22:08:25.802774+0000","last_unstale":"2023-01-24T22:08:25.805162+0000","last_undegraded":"2023-01-24T22:08:25.805162+0000","last_fullsized":"2023-01-24T22:08:25.805162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.805129+0000","last_change":"2023-01-24T22:08:25.802558+0000","last_active":"2023-01-24T22:08:25.805129+0000","last_peered":"2023-01-24T22:08:25.805129+0000","last_clean":"2023-01-24T22:08:25.805129+0000","last_became_active":"2023-01-24T22:08:25.802204+0000","last_became_peered":"2023-01-24T22:08:25.802204+0000","last_unstale":"2023-01-24T22:08:25.805129+0000","last_undegraded":"2023-01-24T22:08:25.805129+0000","last_fullsized":"2023-01-24T22:08:25.805129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.157213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803654+0000","last_change":"2023-01-24T22:08:25.801490+0000","last_active":"2023-01-24T22:08:25.803654+0000","last_peered":"2023-01-24T22:08:25.803654+0000","last_clean":"2023-01-24T22:08:25.803654+0000","last_became_active":"2023-01-24T22:08:25.801046+0000","last_became_peered":"2023-01-24T22:08:25.801046+0000","last_unstale":"2023-01-24T22:08:25.803654+0000","last_undegraded":"2023-01-24T22:08:25.803654+0000","last_fullsized":"2023-01-24T22:08:25.803654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:42.654737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804977+0000","last_change":"2023-01-24T22:08:25.802315+0000","last_active":"2023-01-24T22:08:25.804977+0000","last_peered":"2023-01-24T22:08:25.804977+0000","last_clean":"2023-01-24T22:08:25.804977+0000","last_became_active":"2023-01-24T22:08:25.802003+0000","last_became_peered":"2023-01-24T22:08:25.802003+0000","last_unstale":"2023-01-24T22:08:25.804977+0000","last_undegraded":"2023-01-24T22:08:25.804977+0000","last_fullsized":"2023-01-24T22:08:25.804977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:28.599628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803128+0000","last_change":"2023-01-24T22:08:25.799680+0000","last_active":"2023-01-24T22:08:25.803128+0000","last_peered":"2023-01-24T22:08:25.803128+0000","last_clean":"2023-01-24T22:08:25.803128+0000","last_became_active":"2023-01-24T22:08:25.799419+0000","last_became_peered":"2023-01-24T22:08:25.799419+0000","last_unstale":"2023-01-24T22:08:25.803128+0000","last_undegraded":"2023-01-24T22:08:25.803128+0000","last_fullsized":"2023-01-24T22:08:25.803128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:43.005484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T22:08:26.948706+0000","last_change":"2023-01-24T22:08:25.799400+0000","last_active":"2023-01-24T22:08:26.948706+0000","last_peered":"2023-01-24T22:08:26.948706+0000","last_clean":"2023-01-24T22:08:26.948706+0000","last_became_active":"2023-01-24T22:08:25.799019+0000","last_became_peered":"2023-01-24T22:08:25.799019+0000","last_unstale":"2023-01-24T22:08:26.948706+0000","last_undegraded":"2023-01-24T22:08:26.948706+0000","last_fullsized":"2023-01-24T22:08:26.948706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804798+0000","last_change":"2023-01-24T22:08:25.803755+0000","last_active":"2023-01-24T22:08:25.804798+0000","last_peered":"2023-01-24T22:08:25.804798+0000","last_clean":"2023-01-24T22:08:25.804798+0000","last_became_active":"2023-01-24T22:08:25.803357+0000","last_became_peered":"2023-01-24T22:08:25.803357+0000","last_unstale":"2023-01-24T22:08:25.804798+0000","last_undegraded":"2023-01-24T22:08:25.804798+0000","last_fullsized":"2023-01-24T22:08:25.804798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:25.269566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804890+0000","last_change":"2023-01-24T22:08:25.804048+0000","last_active":"2023-01-24T22:08:25.804890+0000","last_peered":"2023-01-24T22:08:25.804890+0000","last_clean":"2023-01-24T22:08:25.804890+0000","last_became_active":"2023-01-24T22:08:25.803718+0000","last_became_peered":"2023-01-24T22:08:25.803718+0000","last_unstale":"2023-01-24T22:08:25.804890+0000","last_undegraded":"2023-01-24T22:08:25.804890+0000","last_fullsized":"2023-01-24T22:08:25.804890+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.933690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804643+0000","last_change":"2023-01-24T22:08:25.802276+0000","last_active":"2023-01-24T22:08:25.804643+0000","last_peered":"2023-01-24T22:08:25.804643+0000","last_clean":"2023-01-24T22:08:25.804643+0000","last_became_active":"2023-01-24T22:08:25.801955+0000","last_became_peered":"2023-01-24T22:08:25.801955+0000","last_unstale":"2023-01-24T22:08:25.804643+0000","last_undegraded":"2023-01-24T22:08:25.804643+0000","last_fullsized":"2023-01-24T22:08:25.804643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:37.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5119999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4610000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.871}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8239999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9550000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T22:08:31.420 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T22:08:31.734 INFO:teuthology.orchestra.run.smithi035.stdout: 2023-01-24T22:08:31.736 INFO:teuthology.orchestra.run.smithi035.stderr:dumped all 2023-01-24T22:08:31.747 INFO:teuthology.orchestra.run.smithi035.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T22:08:30.700686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44748,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":40640,"kb_avail":754929972,"statfs":{"total":773094113280,"available":773048291328,"internally_reserved":0,"allocated":2777088,"data_stored":1951552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41615360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.805162+0000","last_change":"2023-01-24T22:08:25.803109+0000","last_active":"2023-01-24T22:08:25.805162+0000","last_peered":"2023-01-24T22:08:25.805162+0000","last_clean":"2023-01-24T22:08:25.805162+0000","last_became_active":"2023-01-24T22:08:25.802774+0000","last_became_peered":"2023-01-24T22:08:25.802774+0000","last_unstale":"2023-01-24T22:08:25.805162+0000","last_undegraded":"2023-01-24T22:08:25.805162+0000","last_fullsized":"2023-01-24T22:08:25.805162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.805129+0000","last_change":"2023-01-24T22:08:25.802558+0000","last_active":"2023-01-24T22:08:25.805129+0000","last_peered":"2023-01-24T22:08:25.805129+0000","last_clean":"2023-01-24T22:08:25.805129+0000","last_became_active":"2023-01-24T22:08:25.802204+0000","last_became_peered":"2023-01-24T22:08:25.802204+0000","last_unstale":"2023-01-24T22:08:25.805129+0000","last_undegraded":"2023-01-24T22:08:25.805129+0000","last_fullsized":"2023-01-24T22:08:25.805129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.157213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803654+0000","last_change":"2023-01-24T22:08:25.801490+0000","last_active":"2023-01-24T22:08:25.803654+0000","last_peered":"2023-01-24T22:08:25.803654+0000","last_clean":"2023-01-24T22:08:25.803654+0000","last_became_active":"2023-01-24T22:08:25.801046+0000","last_became_peered":"2023-01-24T22:08:25.801046+0000","last_unstale":"2023-01-24T22:08:25.803654+0000","last_undegraded":"2023-01-24T22:08:25.803654+0000","last_fullsized":"2023-01-24T22:08:25.803654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:42.654737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804977+0000","last_change":"2023-01-24T22:08:25.802315+0000","last_active":"2023-01-24T22:08:25.804977+0000","last_peered":"2023-01-24T22:08:25.804977+0000","last_clean":"2023-01-24T22:08:25.804977+0000","last_became_active":"2023-01-24T22:08:25.802003+0000","last_became_peered":"2023-01-24T22:08:25.802003+0000","last_unstale":"2023-01-24T22:08:25.804977+0000","last_undegraded":"2023-01-24T22:08:25.804977+0000","last_fullsized":"2023-01-24T22:08:25.804977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:28.599628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.803128+0000","last_change":"2023-01-24T22:08:25.799680+0000","last_active":"2023-01-24T22:08:25.803128+0000","last_peered":"2023-01-24T22:08:25.803128+0000","last_clean":"2023-01-24T22:08:25.803128+0000","last_became_active":"2023-01-24T22:08:25.799419+0000","last_became_peered":"2023-01-24T22:08:25.799419+0000","last_unstale":"2023-01-24T22:08:25.803128+0000","last_undegraded":"2023-01-24T22:08:25.803128+0000","last_fullsized":"2023-01-24T22:08:25.803128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:43.005484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T22:08:26.948706+0000","last_change":"2023-01-24T22:08:25.799400+0000","last_active":"2023-01-24T22:08:26.948706+0000","last_peered":"2023-01-24T22:08:26.948706+0000","last_clean":"2023-01-24T22:08:26.948706+0000","last_became_active":"2023-01-24T22:08:25.799019+0000","last_became_peered":"2023-01-24T22:08:25.799019+0000","last_unstale":"2023-01-24T22:08:26.948706+0000","last_undegraded":"2023-01-24T22:08:26.948706+0000","last_fullsized":"2023-01-24T22:08:26.948706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.284179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804798+0000","last_change":"2023-01-24T22:08:25.803755+0000","last_active":"2023-01-24T22:08:25.804798+0000","last_peered":"2023-01-24T22:08:25.804798+0000","last_clean":"2023-01-24T22:08:25.804798+0000","last_became_active":"2023-01-24T22:08:25.803357+0000","last_became_peered":"2023-01-24T22:08:25.803357+0000","last_unstale":"2023-01-24T22:08:25.804798+0000","last_undegraded":"2023-01-24T22:08:25.804798+0000","last_fullsized":"2023-01-24T22:08:25.804798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:25.269566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804890+0000","last_change":"2023-01-24T22:08:25.804048+0000","last_active":"2023-01-24T22:08:25.804890+0000","last_peered":"2023-01-24T22:08:25.804890+0000","last_clean":"2023-01-24T22:08:25.804890+0000","last_became_active":"2023-01-24T22:08:25.803718+0000","last_became_peered":"2023-01-24T22:08:25.803718+0000","last_unstale":"2023-01-24T22:08:25.804890+0000","last_undegraded":"2023-01-24T22:08:25.804890+0000","last_fullsized":"2023-01-24T22:08:25.804890+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:52.933690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T22:08:25.804643+0000","last_change":"2023-01-24T22:08:25.802276+0000","last_active":"2023-01-24T22:08:25.804643+0000","last_peered":"2023-01-24T22:08:25.804643+0000","last_clean":"2023-01-24T22:08:25.804643+0000","last_became_active":"2023-01-24T22:08:25.801955+0000","last_became_peered":"2023-01-24T22:08:25.801955+0000","last_unstale":"2023-01-24T22:08:25.804643+0000","last_undegraded":"2023-01-24T22:08:25.804643+0000","last_fullsized":"2023-01-24T22:08:25.804643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T22:08:24.784519+0000","last_clean_scrub_stamp":"2023-01-24T22:08:24.784519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T22:09:37.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5119999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4610000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6492,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94365348,"statfs":{"total":96636764160,"available":96630116352,"internally_reserved":0,"allocated":679936,"data_stored":574524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.871}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8239999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9550000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T22:08:31.749 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T22:08:31.749 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-24T22:08:32.067 INFO:teuthology.orchestra.run.smithi035.stdout:epoch 17 2023-01-24T22:08:32.067 INFO:teuthology.orchestra.run.smithi035.stdout:fsid 9ce212a6-06d7-42fa-957c-35b0162a9d82 2023-01-24T22:08:32.068 INFO:teuthology.orchestra.run.smithi035.stdout:created 2023-01-24T22:08:13.408713+0000 2023-01-24T22:08:32.068 INFO:teuthology.orchestra.run.smithi035.stdout:modified 2023-01-24T22:08:26.795296+0000 2023-01-24T22:08:32.068 INFO:teuthology.orchestra.run.smithi035.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-24T22:08:32.068 INFO:teuthology.orchestra.run.smithi035.stdout:crush_version 4 2023-01-24T22:08:32.069 INFO:teuthology.orchestra.run.smithi035.stdout:full_ratio 0.95 2023-01-24T22:08:32.069 INFO:teuthology.orchestra.run.smithi035.stdout:backfillfull_ratio 0.9 2023-01-24T22:08:32.069 INFO:teuthology.orchestra.run.smithi035.stdout:nearfull_ratio 0.85 2023-01-24T22:08:32.069 INFO:teuthology.orchestra.run.smithi035.stdout:require_min_compat_client luminous 2023-01-24T22:08:32.069 INFO:teuthology.orchestra.run.smithi035.stdout:min_compat_client jewel 2023-01-24T22:08:32.070 INFO:teuthology.orchestra.run.smithi035.stdout:require_osd_release quincy 2023-01-24T22:08:32.070 INFO:teuthology.orchestra.run.smithi035.stdout:stretch_mode_enabled false 2023-01-24T22:08:32.070 INFO:teuthology.orchestra.run.smithi035.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-24T22:08:32.070 INFO:teuthology.orchestra.run.smithi035.stdout:pool 2 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-24T22:08:32.070 INFO:teuthology.orchestra.run.smithi035.stdout:max_osd 8 2023-01-24T22:08:32.071 INFO:teuthology.orchestra.run.smithi035.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6800/82448 v1:172.21.15.35:6801/82448 exists,up 9a8a7db9-2c26-4445-b328-5a965fbc9cf9 2023-01-24T22:08:32.071 INFO:teuthology.orchestra.run.smithi035.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6804/82449 v1:172.21.15.35:6805/82449 exists,up 361be473-2862-4c57-b199-4a3f3a713e00 2023-01-24T22:08:32.071 INFO:teuthology.orchestra.run.smithi035.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6812/82474 v1:172.21.15.35:6813/82474 exists,up fde90b4f-fe4b-4d22-ba26-14f9cc411262 2023-01-24T22:08:32.071 INFO:teuthology.orchestra.run.smithi035.stdout:osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6808/82464 v1:172.21.15.35:6809/82464 exists,up bef50b21-75dd-4460-89f9-ce28580655ab 2023-01-24T22:08:32.071 INFO:teuthology.orchestra.run.smithi035.stdout:osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6800/82427 v1:172.21.15.72:6801/82427 exists,up 3fe3785d-179e-4164-8914-8d7f0700661d 2023-01-24T22:08:32.072 INFO:teuthology.orchestra.run.smithi035.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6808/82457 v1:172.21.15.72:6809/82457 exists,up d32d4dbc-59f9-41e1-b581-c539f53482c2 2023-01-24T22:08:32.072 INFO:teuthology.orchestra.run.smithi035.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6804/82428 v1:172.21.15.72:6805/82428 exists,up 2ca4febc-902c-419b-b1dc-a56ef94a80fc 2023-01-24T22:08:32.072 INFO:teuthology.orchestra.run.smithi035.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6812/82458 v1:172.21.15.72:6813/82458 exists,up c53abbcf-8848-4514-8f73-151f728381ff 2023-01-24T22:08:32.081 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6800/82448 v1:172.21.15.35:6801/82448 exists,up 9a8a7db9-2c26-4445-b328-5a965fbc9cf9', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6804/82449 v1:172.21.15.35:6805/82449 exists,up 361be473-2862-4c57-b199-4a3f3a713e00', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6812/82474 v1:172.21.15.35:6813/82474 exists,up fde90b4f-fe4b-4d22-ba26-14f9cc411262', 'osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.35:6808/82464 v1:172.21.15.35:6809/82464 exists,up bef50b21-75dd-4460-89f9-ce28580655ab', 'osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6800/82427 v1:172.21.15.72:6801/82427 exists,up 3fe3785d-179e-4164-8914-8d7f0700661d', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6808/82457 v1:172.21.15.72:6809/82457 exists,up d32d4dbc-59f9-41e1-b581-c539f53482c2', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6804/82428 v1:172.21.15.72:6805/82428 exists,up 2ca4febc-902c-419b-b1dc-a56ef94a80fc', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6812/82458 v1:172.21.15.72:6813/82458 exists,up c53abbcf-8848-4514-8f73-151f728381ff'] 2023-01-24T22:08:32.081 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-24T22:08:32.200 INFO:teuthology.orchestra.run.smithi035.stdout:{ 2023-01-24T22:08:32.200 INFO:teuthology.orchestra.run.smithi035.stdout: "name": "mon.a", 2023-01-24T22:08:32.204 INFO:teuthology.orchestra.run.smithi035.stdout: "cluster": "ceph", 2023-01-24T22:08:32.204 INFO:teuthology.orchestra.run.smithi035.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-24T22:08:32.204 INFO:teuthology.orchestra.run.smithi035.stdout: "admin_socket_mode": "", 2023-01-24T22:08:32.205 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-24T22:08:32.205 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_client_required": "cephx, none", 2023-01-24T22:08:32.205 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_cluster_required": "cephx", 2023-01-24T22:08:32.206 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_debug": "true", 2023-01-24T22:08:32.206 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-24T22:08:32.206 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-24T22:08:32.206 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_service_required": "cephx", 2023-01-24T22:08:32.207 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-24T22:08:32.207 INFO:teuthology.orchestra.run.smithi035.stdout: "auth_supported": "cephx", 2023-01-24T22:08:32.207 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_aio": "true", 2023-01-24T22:08:32.207 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-24T22:08:32.207 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_aio_poll_ms": "250", 2023-01-24T22:08:32.208 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_aio_reap_max": "16", 2023-01-24T22:08:32.208 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_async_discard": "false", 2023-01-24T22:08:32.208 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_block_size": "4096", 2023-01-24T22:08:32.208 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_debug_aio": "false", 2023-01-24T22:08:32.208 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-24T22:08:32.209 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-24T22:08:32.209 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_debug_inflight_ios": "false", 2023-01-24T22:08:32.209 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_enable_discard": "false", 2023-01-24T22:08:32.209 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_flock_retry": "3", 2023-01-24T22:08:32.209 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-24T22:08:32.210 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_inject_crash": "0", 2023-01-24T22:08:32.210 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-24T22:08:32.210 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_ioring": "false", 2023-01-24T22:08:32.210 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_ioring_hipri": "false", 2023-01-24T22:08:32.210 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-24T22:08:32.211 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-24T22:08:32.211 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-24T22:08:32.211 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-24T22:08:32.211 INFO:teuthology.orchestra.run.smithi035.stdout: "bdev_type": "", 2023-01-24T22:08:32.211 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_alloc_size": "1048576", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_allocator": "hybrid", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_buffered_io": "true", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_check_for_zeros": "false", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-24T22:08:32.212 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_compact_log_sync": "false", 2023-01-24T22:08:32.213 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-24T22:08:32.213 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-24T22:08:32.213 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-24T22:08:32.213 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_max_log_runway": "4194304", 2023-01-24T22:08:32.213 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_max_prefetch": "1048576", 2023-01-24T22:08:32.214 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_min_flush_size": "524288", 2023-01-24T22:08:32.214 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_min_log_runway": "1048576", 2023-01-24T22:08:32.214 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_replay_recovery": "false", 2023-01-24T22:08:32.214 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-24T22:08:32.214 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-24T22:08:32.215 INFO:teuthology.orchestra.run.smithi035.stdout: "bluefs_sync_write": "false", 2023-01-24T22:08:32.215 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-24T22:08:32.215 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-24T22:08:32.215 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-24T22:08:32.215 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_allocation_from_file": "true", 2023-01-24T22:08:32.216 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_allocator": "hybrid", 2023-01-24T22:08:32.216 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-24T22:08:32.216 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-24T22:08:32.216 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-24T22:08:32.216 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-24T22:08:32.217 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-24T22:08:32.217 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-24T22:08:32.217 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-24T22:08:32.217 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_create": "true", 2023-01-24T22:08:32.217 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_db_create": "false", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_db_path": "", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_db_size": "0", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_path": "", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_preallocate_file": "false", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_size": "107374182400", 2023-01-24T22:08:32.218 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_wal_create": "false", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_wal_path": "", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_block_wal_size": "100663296", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bluefs": "true", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-24T22:08:32.219 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-24T22:08:32.220 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-24T22:08:32.220 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T22:08:32.220 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T22:08:32.220 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-24T22:08:32.220 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T22:08:32.221 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_autotune": "true", 2023-01-24T22:08:32.221 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-24T22:08:32.221 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-24T22:08:32.221 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-24T22:08:32.221 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_size": "0", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-24T22:08:32.222 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cache_type": "2q", 2023-01-24T22:08:32.223 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-24T22:08:32.223 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_clone_cow": "true", 2023-01-24T22:08:32.223 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-24T22:08:32.223 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-24T22:08:32.223 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-24T22:08:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-24T22:08:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-24T22:08:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-24T22:08:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-24T22:08:32.224 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_mode": "none", 2023-01-24T22:08:32.225 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-24T22:08:32.225 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_csum_type": "crc32c", 2023-01-24T22:08:32.225 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-24T22:08:32.225 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_freelist": "false", 2023-01-24T22:08:32.226 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-24T22:08:32.226 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-24T22:08:32.226 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-24T22:08:32.226 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-24T22:08:32.226 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-24T22:08:32.227 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-24T22:08:32.228 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-24T22:08:32.228 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-24T22:08:32.228 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_small_allocations": "0", 2023-01-24T22:08:32.228 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-24T22:08:32.228 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_default_buffered_read": "true", 2023-01-24T22:08:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_default_buffered_write": "false", 2023-01-24T22:08:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-24T22:08:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-24T22:08:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-24T22:08:32.229 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-24T22:08:32.230 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-24T22:08:32.230 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-24T22:08:32.230 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-24T22:08:32.230 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-24T22:08:32.230 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fail_eio": "false", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-24T22:08:32.231 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-24T22:08:32.232 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_mount": "false", 2023-01-24T22:08:32.232 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-24T22:08:32.232 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_umount": "false", 2023-01-24T22:08:32.232 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-24T22:08:32.233 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-24T22:08:32.233 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-24T22:08:32.233 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-24T22:08:32.233 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-24T22:08:32.233 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-24T22:08:32.234 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-24T22:08:32.234 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-24T22:08:32.234 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_ignore_data_csum": "false", 2023-01-24T22:08:32.234 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-24T22:08:32.234 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_log_op_age": "5.000000", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_alloc_size": "0", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_blob_size": "0", 2023-01-24T22:08:32.235 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-24T22:08:32.236 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-24T22:08:32.236 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-24T22:08:32.236 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_max_deferred_txc": "32", 2023-01-24T22:08:32.236 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_min_alloc_size": "0", 2023-01-24T22:08:32.236 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_nid_prealloc": "1024", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-24T22:08:32.237 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-24T22:08:32.238 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_retry_disk_reads": "3", 2023-01-24T22:08:32.238 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_rocksdb_cf": "true", 2023-01-24T22:08:32.238 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-24T22:08:32.238 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2023-01-24T22:08:32.238 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-24T22:08:32.239 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-24T22:08:32.239 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-24T22:08:32.239 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-24T22:08:32.239 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_spdk_mem": "512", 2023-01-24T22:08:32.239 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-24T22:08:32.240 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-24T22:08:32.240 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-24T22:08:32.240 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-24T22:08:32.240 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-24T22:08:32.240 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-24T22:08:32.241 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-24T22:08:32.241 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_tracing": "false", 2023-01-24T22:08:32.241 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-24T22:08:32.241 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-24T22:08:32.241 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-24T22:08:32.242 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-24T22:08:32.242 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-24T22:08:32.242 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-24T22:08:32.242 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-24T22:08:32.242 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "bluestore_zero_block_detection": "false", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-24T22:08:32.243 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-24T22:08:32.244 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-24T22:08:32.244 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-24T22:08:32.244 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-24T22:08:32.244 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-24T22:08:32.245 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_cluster_require_signatures": "false", 2023-01-24T22:08:32.246 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_cluster_require_version": "2", 2023-01-24T22:08:32.246 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_require_signatures": "false", 2023-01-24T22:08:32.246 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_require_version": "2", 2023-01-24T22:08:32.246 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_service_require_signatures": "false", 2023-01-24T22:08:32.246 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_service_require_version": "2", 2023-01-24T22:08:32.247 INFO:teuthology.orchestra.run.smithi035.stdout: "cephx_sign_messages": "true", 2023-01-24T22:08:32.247 INFO:teuthology.orchestra.run.smithi035.stdout: "chdir": "", 2023-01-24T22:08:32.247 INFO:teuthology.orchestra.run.smithi035.stdout: "client_acl_type": "", 2023-01-24T22:08:32.247 INFO:teuthology.orchestra.run.smithi035.stdout: "client_asio_thread_count": "2", 2023-01-24T22:08:32.247 INFO:teuthology.orchestra.run.smithi035.stdout: "client_cache_mid": "0.750000", 2023-01-24T22:08:32.248 INFO:teuthology.orchestra.run.smithi035.stdout: "client_cache_size": "16384", 2023-01-24T22:08:32.248 INFO:teuthology.orchestra.run.smithi035.stdout: "client_caps_release_delay": "5", 2023-01-24T22:08:32.248 INFO:teuthology.orchestra.run.smithi035.stdout: "client_check_pool_perm": "true", 2023-01-24T22:08:32.248 INFO:teuthology.orchestra.run.smithi035.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-24T22:08:32.248 INFO:teuthology.orchestra.run.smithi035.stdout: "client_debug_force_sync_read": "false", 2023-01-24T22:08:32.249 INFO:teuthology.orchestra.run.smithi035.stdout: "client_debug_getattr_caps": "false", 2023-01-24T22:08:32.249 INFO:teuthology.orchestra.run.smithi035.stdout: "client_debug_inject_tick_delay": "0", 2023-01-24T22:08:32.249 INFO:teuthology.orchestra.run.smithi035.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-24T22:08:32.249 INFO:teuthology.orchestra.run.smithi035.stdout: "client_die_on_failed_remount": "false", 2023-01-24T22:08:32.249 INFO:teuthology.orchestra.run.smithi035.stdout: "client_dirsize_rbytes": "true", 2023-01-24T22:08:32.250 INFO:teuthology.orchestra.run.smithi035.stdout: "client_force_lazyio": "false", 2023-01-24T22:08:32.250 INFO:teuthology.orchestra.run.smithi035.stdout: "client_fs": "", 2023-01-24T22:08:32.250 INFO:teuthology.orchestra.run.smithi035.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-24T22:08:32.250 INFO:teuthology.orchestra.run.smithi035.stdout: "client_inject_release_failure": "false", 2023-01-24T22:08:32.251 INFO:teuthology.orchestra.run.smithi035.stdout: "client_max_inline_size": "4096", 2023-01-24T22:08:32.251 INFO:teuthology.orchestra.run.smithi035.stdout: "client_mds_namespace": "", 2023-01-24T22:08:32.251 INFO:teuthology.orchestra.run.smithi035.stdout: "client_metadata": "", 2023-01-24T22:08:32.251 INFO:teuthology.orchestra.run.smithi035.stdout: "client_mount_gid": "-1", 2023-01-24T22:08:32.251 INFO:teuthology.orchestra.run.smithi035.stdout: "client_mount_timeout": "300", 2023-01-24T22:08:32.252 INFO:teuthology.orchestra.run.smithi035.stdout: "client_mount_uid": "-1", 2023-01-24T22:08:32.252 INFO:teuthology.orchestra.run.smithi035.stdout: "client_mountpoint": "/", 2023-01-24T22:08:32.252 INFO:teuthology.orchestra.run.smithi035.stdout: "client_notify_timeout": "10", 2023-01-24T22:08:32.252 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc": "true", 2023-01-24T22:08:32.252 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc_max_dirty": "104857600", 2023-01-24T22:08:32.253 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-24T22:08:32.253 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc_max_objects": "1000", 2023-01-24T22:08:32.253 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc_size": "209715200", 2023-01-24T22:08:32.253 INFO:teuthology.orchestra.run.smithi035.stdout: "client_oc_target_dirty": "8388608", 2023-01-24T22:08:32.253 INFO:teuthology.orchestra.run.smithi035.stdout: "client_permissions": "true", 2023-01-24T22:08:32.254 INFO:teuthology.orchestra.run.smithi035.stdout: "client_quota": "true", 2023-01-24T22:08:32.254 INFO:teuthology.orchestra.run.smithi035.stdout: "client_quota_df": "true", 2023-01-24T22:08:32.254 INFO:teuthology.orchestra.run.smithi035.stdout: "client_readahead_max_bytes": "0", 2023-01-24T22:08:32.254 INFO:teuthology.orchestra.run.smithi035.stdout: "client_readahead_max_periods": "4", 2023-01-24T22:08:32.254 INFO:teuthology.orchestra.run.smithi035.stdout: "client_readahead_min": "131072", 2023-01-24T22:08:32.255 INFO:teuthology.orchestra.run.smithi035.stdout: "client_reconnect_stale": "false", 2023-01-24T22:08:32.255 INFO:teuthology.orchestra.run.smithi035.stdout: "client_shutdown_timeout": "30", 2023-01-24T22:08:32.255 INFO:teuthology.orchestra.run.smithi035.stdout: "client_snapdir": ".snap", 2023-01-24T22:08:32.255 INFO:teuthology.orchestra.run.smithi035.stdout: "client_tick_interval": "1", 2023-01-24T22:08:32.255 INFO:teuthology.orchestra.run.smithi035.stdout: "client_trace": "", 2023-01-24T22:08:32.256 INFO:teuthology.orchestra.run.smithi035.stdout: "client_try_dentry_invalidate": "false", 2023-01-24T22:08:32.256 INFO:teuthology.orchestra.run.smithi035.stdout: "client_use_faked_inos": "false", 2023-01-24T22:08:32.256 INFO:teuthology.orchestra.run.smithi035.stdout: "client_use_random_mds": "false", 2023-01-24T22:08:32.256 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_graylog": "false", 2023-01-24T22:08:32.256 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-24T22:08:32.257 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_graylog_port": "12201", 2023-01-24T22:08:32.257 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_monitors": "default=true", 2023-01-24T22:08:32.257 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_syslog": "false", 2023-01-24T22:08:32.257 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-24T22:08:32.257 INFO:teuthology.orchestra.run.smithi035.stdout: "clog_to_syslog_level": "info", 2023-01-24T22:08:32.258 INFO:teuthology.orchestra.run.smithi035.stdout: "cluster_addr": "-", 2023-01-24T22:08:32.258 INFO:teuthology.orchestra.run.smithi035.stdout: "cluster_network": "", 2023-01-24T22:08:32.258 INFO:teuthology.orchestra.run.smithi035.stdout: "cluster_network_interface": "", 2023-01-24T22:08:32.258 INFO:teuthology.orchestra.run.smithi035.stdout: "compressor_zlib_isal": "false", 2023-01-24T22:08:32.258 INFO:teuthology.orchestra.run.smithi035.stdout: "compressor_zlib_level": "5", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "compressor_zlib_winsize": "-15", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "compressor_zstd_level": "1", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "crimson_alien_op_num_threads": "6", 2023-01-24T22:08:32.259 INFO:teuthology.orchestra.run.smithi035.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-24T22:08:32.260 INFO:teuthology.orchestra.run.smithi035.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-24T22:08:32.260 INFO:teuthology.orchestra.run.smithi035.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-24T22:08:32.260 INFO:teuthology.orchestra.run.smithi035.stdout: "crush_location": "", 2023-01-24T22:08:32.260 INFO:teuthology.orchestra.run.smithi035.stdout: "crush_location_hook": "", 2023-01-24T22:08:32.260 INFO:teuthology.orchestra.run.smithi035.stdout: "crush_location_hook_timeout": "10", 2023-01-24T22:08:32.261 INFO:teuthology.orchestra.run.smithi035.stdout: "daemonize": "false", 2023-01-24T22:08:32.261 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_alienstore": "0/5", 2023-01-24T22:08:32.261 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_allow_any_pool_priority": "false", 2023-01-24T22:08:32.261 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_asok": "1/5", 2023-01-24T22:08:32.261 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_asok_assert_abort": "false", 2023-01-24T22:08:32.262 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_asserts_on_shutdown": "true", 2023-01-24T22:08:32.262 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_auth": "20/20", 2023-01-24T22:08:32.262 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_bdev": "1/3", 2023-01-24T22:08:32.262 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_bluefs": "1/5", 2023-01-24T22:08:32.262 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_bluestore": "1/5", 2023-01-24T22:08:32.263 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_buffer": "0/1", 2023-01-24T22:08:32.263 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_ceph_exporter": "1/5", 2023-01-24T22:08:32.263 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_cephfs_mirror": "0/5", 2023-01-24T22:08:32.263 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_cephsqlite": "0/5", 2023-01-24T22:08:32.263 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_civetweb": "1/10", 2023-01-24T22:08:32.264 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_client": "0/5", 2023-01-24T22:08:32.264 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_compressor": "1/5", 2023-01-24T22:08:32.264 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_context": "0/1", 2023-01-24T22:08:32.264 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_crush": "1/1", 2023-01-24T22:08:32.264 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_crypto": "1/5", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_deliberately_leak_memory": "false", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_disable_randomized_ping": "false", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_dpdk": "1/5", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_eventtrace": "1/5", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_filer": "0/1", 2023-01-24T22:08:32.265 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_filestore": "1/3", 2023-01-24T22:08:32.266 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_finisher": "1/1", 2023-01-24T22:08:32.266 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_fuse": "1/5", 2023-01-24T22:08:32.266 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_heartbeat_testing_span": "0", 2023-01-24T22:08:32.266 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_heartbeatmap": "1/5", 2023-01-24T22:08:32.266 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-24T22:08:32.267 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_javaclient": "1/5", 2023-01-24T22:08:32.267 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_journal": "1/3", 2023-01-24T22:08:32.267 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_journaler": "0/5", 2023-01-24T22:08:32.267 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_kstore": "1/5", 2023-01-24T22:08:32.267 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_leveldb": "4/5", 2023-01-24T22:08:32.268 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_lockdep": "0/1", 2023-01-24T22:08:32.268 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mclock": "1/5", 2023-01-24T22:08:32.268 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds": "1/5", 2023-01-24T22:08:32.269 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds_balancer": "1/5", 2023-01-24T22:08:32.269 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds_locker": "1/5", 2023-01-24T22:08:32.269 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds_log": "1/5", 2023-01-24T22:08:32.269 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds_log_expire": "1/5", 2023-01-24T22:08:32.269 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mds_migrator": "1/5", 2023-01-24T22:08:32.270 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_memdb": "4/5", 2023-01-24T22:08:32.270 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mgr": "2/5", 2023-01-24T22:08:32.270 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mgrc": "1/5", 2023-01-24T22:08:32.270 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_mon": "20/20", 2023-01-24T22:08:32.270 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_monc": "0/10", 2023-01-24T22:08:32.271 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_ms": "1/1", 2023-01-24T22:08:32.271 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_none": "0/5", 2023-01-24T22:08:32.271 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_objclass": "0/5", 2023-01-24T22:08:32.271 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_objectcacher": "0/5", 2023-01-24T22:08:32.271 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_objecter": "0/1", 2023-01-24T22:08:32.272 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_optracker": "0/5", 2023-01-24T22:08:32.272 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_osd": "1/5", 2023-01-24T22:08:32.272 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_paxos": "20/20", 2023-01-24T22:08:32.272 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_perfcounter": "1/5", 2023-01-24T22:08:32.272 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_prioritycache": "1/5", 2023-01-24T22:08:32.273 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rados": "0/5", 2023-01-24T22:08:32.273 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rbd": "0/5", 2023-01-24T22:08:32.273 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rbd_mirror": "0/5", 2023-01-24T22:08:32.273 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rbd_pwl": "0/5", 2023-01-24T22:08:32.273 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rbd_replay": "0/5", 2023-01-24T22:08:32.274 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_refs": "0/0", 2023-01-24T22:08:32.274 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_reserver": "1/1", 2023-01-24T22:08:32.274 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rgw": "1/5", 2023-01-24T22:08:32.274 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rgw_datacache": "1/5", 2023-01-24T22:08:32.274 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rgw_sync": "1/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_rocksdb": "4/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore": "0/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_cache": "0/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_cleaner": "0/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_device": "0/5", 2023-01-24T22:08:32.275 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_journal": "0/5", 2023-01-24T22:08:32.276 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_lba": "0/5", 2023-01-24T22:08:32.276 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_odata": "0/5", 2023-01-24T22:08:32.276 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_omap": "0/5", 2023-01-24T22:08:32.276 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_onode": "0/5", 2023-01-24T22:08:32.276 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_seastore_tm": "0/5", 2023-01-24T22:08:32.277 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_striper": "0/1", 2023-01-24T22:08:32.277 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_test": "0/5", 2023-01-24T22:08:32.277 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_throttle": "1/1", 2023-01-24T22:08:32.277 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_timer": "0/1", 2023-01-24T22:08:32.277 INFO:teuthology.orchestra.run.smithi035.stdout: "debug_tp": "0/5", 2023-01-24T22:08:32.278 INFO:teuthology.orchestra.run.smithi035.stdout: "device_failure_prediction_mode": "none", 2023-01-24T22:08:32.278 INFO:teuthology.orchestra.run.smithi035.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-24T22:08:32.278 INFO:teuthology.orchestra.run.smithi035.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-24T22:08:32.278 INFO:teuthology.orchestra.run.smithi035.stdout: "err_to_graylog": "false", 2023-01-24T22:08:32.278 INFO:teuthology.orchestra.run.smithi035.stdout: "err_to_journald": "false", 2023-01-24T22:08:32.279 INFO:teuthology.orchestra.run.smithi035.stdout: "err_to_stderr": "true", 2023-01-24T22:08:32.279 INFO:teuthology.orchestra.run.smithi035.stdout: "err_to_syslog": "false", 2023-01-24T22:08:32.279 INFO:teuthology.orchestra.run.smithi035.stdout: "event_tracing": "false", 2023-01-24T22:08:32.279 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_addr": "0.0.0.0", 2023-01-24T22:08:32.279 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_http_port": "9926", 2023-01-24T22:08:32.280 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_prio_limit": "5", 2023-01-24T22:08:32.280 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-24T22:08:32.280 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_sort_metrics": "true", 2023-01-24T22:08:32.280 INFO:teuthology.orchestra.run.smithi035.stdout: "exporter_stats_period": "5", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "fake_statfs_for_testing": "0", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "fatal_signal_handlers": "true", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "filer_max_purge_ops": "10", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "filer_max_truncate_ops": "128", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_apply_finisher_threads": "1", 2023-01-24T22:08:32.281 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_blackhole": "false", 2023-01-24T22:08:32.282 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_btrfs_clone_range": "true", 2023-01-24T22:08:32.282 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_btrfs_snap": "true", 2023-01-24T22:08:32.282 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_caller_concurrency": "10", 2023-01-24T22:08:32.282 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_collect_device_partition_information": "true", 2023-01-24T22:08:32.282 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_commit_timeout": "600.000000", 2023-01-24T22:08:32.283 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_debug_inject_read_err": "false", 2023-01-24T22:08:32.283 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_debug_omap_check": "false", 2023-01-24T22:08:32.283 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-24T22:08:32.283 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_debug_verify_split": "false", 2023-01-24T22:08:32.283 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_dump_file": "", 2023-01-24T22:08:32.284 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-24T22:08:32.284 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-24T22:08:32.284 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fadvise": "true", 2023-01-24T22:08:32.284 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fail_eio": "true", 2023-01-24T22:08:32.284 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fd_cache_shards": "16", 2023-01-24T22:08:32.285 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fd_cache_size": "128", 2023-01-24T22:08:32.285 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fiemap": "false", 2023-01-24T22:08:32.285 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fiemap_threshold": "4096", 2023-01-24T22:08:32.285 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-24T22:08:32.285 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-24T22:08:32.286 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_inject_stall": "0", 2023-01-24T22:08:32.286 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_journal_parallel": "false", 2023-01-24T22:08:32.286 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_journal_trailing": "false", 2023-01-24T22:08:32.286 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_journal_writeahead": "false", 2023-01-24T22:08:32.286 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_kill_at": "0", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-24T22:08:32.287 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattrs": "0", 2023-01-24T22:08:32.288 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-24T22:08:32.288 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-24T22:08:32.288 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-24T22:08:32.288 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-24T22:08:32.288 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_xattr_value_size": "0", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_merge_threshold": "-10", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-24T22:08:32.289 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_odsync_write": "false", 2023-01-24T22:08:32.290 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_omap_backend": "rocksdb", 2023-01-24T22:08:32.290 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_omap_backend_path": "", 2023-01-24T22:08:32.290 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-24T22:08:32.290 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-24T22:08:32.291 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-24T22:08:32.291 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_op_thread_timeout": "60", 2023-01-24T22:08:32.291 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_op_threads": "2", 2023-01-24T22:08:32.291 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_punch_hole": "false", 2023-01-24T22:08:32.291 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-24T22:08:32.292 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-24T22:08:32.292 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-24T22:08:32.292 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-24T22:08:32.292 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-24T22:08:32.292 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-24T22:08:32.293 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-24T22:08:32.293 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-24T22:08:32.293 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-24T22:08:32.293 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_queue_max_ops": "50", 2023-01-24T22:08:32.293 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_seek_data_hole": "false", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_sloppy_crc": "false", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_splice": "false", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_split_multiple": "2", 2023-01-24T22:08:32.294 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_split_rand_factor": "20", 2023-01-24T22:08:32.295 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_update_to": "1000", 2023-01-24T22:08:32.295 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-24T22:08:32.295 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-24T22:08:32.295 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-24T22:08:32.295 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-24T22:08:32.296 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-24T22:08:32.296 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-24T22:08:32.296 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_enable": "true", 2023-01-24T22:08:32.296 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-24T22:08:32.296 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-24T22:08:32.297 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-24T22:08:32.297 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-24T22:08:32.297 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-24T22:08:32.297 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-24T22:08:32.297 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_xfs_extsize": "false", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "filestore_zfs_snap": "false", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "fio_dir": "/tmp/fio", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "fsid": "9ce212a6-06d7-42fa-957c-35b0162a9d82", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_allow_other": "true", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_atomic_o_trunc": "true", 2023-01-24T22:08:32.298 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_big_writes": "true", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_debug": "false", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_default_permissions": "false", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_disable_pagecache": "false", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_max_write": "0", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_multithreaded": "true", 2023-01-24T22:08:32.299 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_require_active_mds": "true", 2023-01-24T22:08:32.300 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_set_user_groups": "true", 2023-01-24T22:08:32.300 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_splice_move": "true", 2023-01-24T22:08:32.300 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_splice_read": "true", 2023-01-24T22:08:32.300 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_splice_write": "true", 2023-01-24T22:08:32.300 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "fuse_use_invalidate_cb": "true", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "gss_target_name": "ceph", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "heartbeat_file": "", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "heartbeat_inject_failure": "0", 2023-01-24T22:08:32.301 INFO:teuthology.orchestra.run.smithi035.stdout: "heartbeat_interval": "5", 2023-01-24T22:08:32.302 INFO:teuthology.orchestra.run.smithi035.stdout: "host": "smithi035", 2023-01-24T22:08:32.302 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-24T22:08:32.302 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-24T22:08:32.302 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-24T22:08:32.302 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-24T22:08:32.303 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-24T22:08:32.303 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-24T22:08:32.303 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-24T22:08:32.303 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-24T22:08:32.304 INFO:teuthology.orchestra.run.smithi035.stdout: "inject_early_sigterm": "false", 2023-01-24T22:08:32.305 INFO:teuthology.orchestra.run.smithi035.stdout: "jaeger_tracing_enable": "false", 2023-01-24T22:08:32.305 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_aio": "true", 2023-01-24T22:08:32.305 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_align_min_size": "65536", 2023-01-24T22:08:32.305 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_block_align": "true", 2023-01-24T22:08:32.305 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_block_size": "4096", 2023-01-24T22:08:32.306 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_dio": "true", 2023-01-24T22:08:32.306 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_discard": "false", 2023-01-24T22:08:32.306 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_force_aio": "false", 2023-01-24T22:08:32.306 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_ignore_corruption": "false", 2023-01-24T22:08:32.306 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_max_write_bytes": "10485760", 2023-01-24T22:08:32.307 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_max_write_entries": "100", 2023-01-24T22:08:32.307 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_replay_from": "0", 2023-01-24T22:08:32.307 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-24T22:08:32.307 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-24T22:08:32.307 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-24T22:08:32.308 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-24T22:08:32.308 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_write_header_frequency": "0", 2023-01-24T22:08:32.308 INFO:teuthology.orchestra.run.smithi035.stdout: "journal_zero_on_create": "false", 2023-01-24T22:08:32.308 INFO:teuthology.orchestra.run.smithi035.stdout: "journaler_prefetch_periods": "10", 2023-01-24T22:08:32.308 INFO:teuthology.orchestra.run.smithi035.stdout: "journaler_prezero_periods": "5", 2023-01-24T22:08:32.309 INFO:teuthology.orchestra.run.smithi035.stdout: "journaler_write_head_interval": "15", 2023-01-24T22:08:32.309 INFO:teuthology.orchestra.run.smithi035.stdout: "key": "", 2023-01-24T22:08:32.309 INFO:teuthology.orchestra.run.smithi035.stdout: "keyfile": "", 2023-01-24T22:08:32.309 INFO:teuthology.orchestra.run.smithi035.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_backend": "rocksdb", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_default_stripe_size": "65536", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_fsck_on_mount": "false", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_max_bytes": "67108864", 2023-01-24T22:08:32.310 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_max_ops": "512", 2023-01-24T22:08:32.311 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_nid_prealloc": "1024", 2023-01-24T22:08:32.311 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_onode_map_size": "1024", 2023-01-24T22:08:32.311 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-24T22:08:32.311 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_sync_submit_transaction": "false", 2023-01-24T22:08:32.311 INFO:teuthology.orchestra.run.smithi035.stdout: "kstore_sync_transaction": "false", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_block_size": "65536", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_bloom_size": "0", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_cache_size": "536870912", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_compact_on_mount": "false", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_compression": "false", 2023-01-24T22:08:32.312 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_log": "", 2023-01-24T22:08:32.313 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-24T22:08:32.313 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_max_open_files": "0", 2023-01-24T22:08:32.313 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_paranoid": "false", 2023-01-24T22:08:32.313 INFO:teuthology.orchestra.run.smithi035.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-24T22:08:32.313 INFO:teuthology.orchestra.run.smithi035.stdout: "librados_thread_count": "2", 2023-01-24T22:08:32.314 INFO:teuthology.orchestra.run.smithi035.stdout: "lockdep": "false", 2023-01-24T22:08:32.314 INFO:teuthology.orchestra.run.smithi035.stdout: "lockdep_force_backtrace": "false", 2023-01-24T22:08:32.314 INFO:teuthology.orchestra.run.smithi035.stdout: "log_coarse_timestamps": "true", 2023-01-24T22:08:32.314 INFO:teuthology.orchestra.run.smithi035.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-24T22:08:32.314 INFO:teuthology.orchestra.run.smithi035.stdout: "log_flush_on_exit": "false", 2023-01-24T22:08:32.315 INFO:teuthology.orchestra.run.smithi035.stdout: "log_graylog_host": "127.0.0.1", 2023-01-24T22:08:32.315 INFO:teuthology.orchestra.run.smithi035.stdout: "log_graylog_port": "12201", 2023-01-24T22:08:32.315 INFO:teuthology.orchestra.run.smithi035.stdout: "log_max_new": "1000", 2023-01-24T22:08:32.315 INFO:teuthology.orchestra.run.smithi035.stdout: "log_max_recent": "10000", 2023-01-24T22:08:32.315 INFO:teuthology.orchestra.run.smithi035.stdout: "log_stderr_prefix": "", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_stop_at_utilization": "0.970000", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_to_file": "true", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_to_graylog": "false", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_to_journald": "false", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_to_stderr": "false", 2023-01-24T22:08:32.316 INFO:teuthology.orchestra.run.smithi035.stdout: "log_to_syslog": "false", 2023-01-24T22:08:32.317 INFO:teuthology.orchestra.run.smithi035.stdout: "max_rotating_auth_attempts": "10", 2023-01-24T22:08:32.317 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_action_on_write_error": "1", 2023-01-24T22:08:32.317 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_alternate_name_max": "8192", 2023-01-24T22:08:32.317 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_asio_thread_count": "2", 2023-01-24T22:08:32.317 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_export_pin": "true", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_fragment_dirs": "true", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_fragment_interval": "5", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-24T22:08:32.318 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_interval": "10", 2023-01-24T22:08:32.319 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_max": "-1", 2023-01-24T22:08:32.319 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_max_until": "-1", 2023-01-24T22:08:32.319 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_merge_size": "50", 2023-01-24T22:08:32.319 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_midchunk": "0.300000", 2023-01-24T22:08:32.319 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-24T22:08:32.320 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_min_start": "0.200000", 2023-01-24T22:08:32.320 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_minchunk": "0.001000", 2023-01-24T22:08:32.320 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_mode": "0", 2023-01-24T22:08:32.320 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_need_max": "1.200000", 2023-01-24T22:08:32.320 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_need_min": "0.800000", 2023-01-24T22:08:32.321 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-24T22:08:32.321 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-24T22:08:32.321 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_split_bits": "3", 2023-01-24T22:08:32.321 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_split_size": "10000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_target_decay": "10.000000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_beacon_grace": "15.000000", 2023-01-24T22:08:32.322 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_beacon_interval": "4.000000", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_mid": "0.700000", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_release_free_interval": "10", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_reservation": "0.050000", 2023-01-24T22:08:32.323 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-24T22:08:32.324 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_trim_interval": "1", 2023-01-24T22:08:32.324 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cache_trim_threshold": "262144", 2023-01-24T22:08:32.324 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-24T22:08:32.324 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-24T22:08:32.324 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_client_prealloc_inos": "1000", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_connect_bootstrapping": "false", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_damage_table_max_entries": "10000", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-24T22:08:32.325 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_debug_auth_pins": "false", 2023-01-24T22:08:32.326 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_debug_frag": "false", 2023-01-24T22:08:32.326 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_debug_scatterstat": "false", 2023-01-24T22:08:32.326 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_debug_subtrees": "false", 2023-01-24T22:08:32.326 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_decay_halflife": "5.000000", 2023-01-24T22:08:32.326 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_default_dir_hash": "2", 2023-01-24T22:08:32.327 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_defer_session_stale": "true", 2023-01-24T22:08:32.327 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_deny_all_reconnect": "false", 2023-01-24T22:08:32.327 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dir_keys_per_op": "16384", 2023-01-24T22:08:32.327 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dir_max_commit_size": "10", 2023-01-24T22:08:32.327 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dir_max_entries": "0", 2023-01-24T22:08:32.328 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-24T22:08:32.328 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-24T22:08:32.328 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dump_cache_on_map": "false", 2023-01-24T22:08:32.329 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-24T22:08:32.329 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-24T22:08:32.329 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_early_reply": "true", 2023-01-24T22:08:32.329 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_enable_op_tracker": "true", 2023-01-24T22:08:32.330 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_enforce_unique_name": "true", 2023-01-24T22:08:32.330 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-24T22:08:32.331 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-24T22:08:32.331 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_export_ephemeral_random": "true", 2023-01-24T22:08:32.331 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-24T22:08:32.331 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-24T22:08:32.331 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-24T22:08:32.332 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-24T22:08:32.332 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-24T22:08:32.332 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_health_summarize_threshold": "10", 2023-01-24T22:08:32.332 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-24T22:08:32.332 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-24T22:08:32.333 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_inject_health_dummy": "false", 2023-01-24T22:08:32.333 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_inject_migrator_session_race": "false", 2023-01-24T22:08:32.333 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-24T22:08:32.333 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_join_fs": "", 2023-01-24T22:08:32.333 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_journal_format": "1", 2023-01-24T22:08:32.334 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_create_at": "0", 2023-01-24T22:08:32.334 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_export_at": "0", 2023-01-24T22:08:32.334 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_import_at": "0", 2023-01-24T22:08:32.334 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_journal_at": "0", 2023-01-24T22:08:32.334 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_journal_expire_at": "0", 2023-01-24T22:08:32.335 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_journal_replay_at": "0", 2023-01-24T22:08:32.335 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_link_at": "0", 2023-01-24T22:08:32.335 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_mdstable_at": "0", 2023-01-24T22:08:32.335 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_openc_at": "0", 2023-01-24T22:08:32.336 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_kill_rename_at": "0", 2023-01-24T22:08:32.336 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_events_per_segment": "1024", 2023-01-24T22:08:32.336 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_max_events": "-1", 2023-01-24T22:08:32.336 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_max_segments": "128", 2023-01-24T22:08:32.336 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_pause": "false", 2023-01-24T22:08:32.337 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_segment_size": "0", 2023-01-24T22:08:32.337 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-24T22:08:32.337 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_log_warn_factor": "2.000000", 2023-01-24T22:08:32.338 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_caps_per_client": "1048576", 2023-01-24T22:08:32.338 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_completed_flushes": "100000", 2023-01-24T22:08:32.339 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_completed_requests": "100000", 2023-01-24T22:08:32.339 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_export_size": "20971520", 2023-01-24T22:08:32.339 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_file_recover": "32", 2023-01-24T22:08:32.340 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_purge_files": "64", 2023-01-24T22:08:32.340 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_purge_ops": "8192", 2023-01-24T22:08:32.341 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-24T22:08:32.341 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-24T22:08:32.341 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-24T22:08:32.341 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_snaps_per_dir": "100", 2023-01-24T22:08:32.342 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-24T22:08:32.342 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_metrics_update_interval": "2", 2023-01-24T22:08:32.342 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_min_caps_per_client": "100", 2023-01-24T22:08:32.342 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_min_caps_working_set": "10000", 2023-01-24T22:08:32.342 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-24T22:08:32.343 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_numa_node": "-1", 2023-01-24T22:08:32.343 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-24T22:08:32.343 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_op_complaint_time": "30.000000", 2023-01-24T22:08:32.343 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_op_history_duration": "600", 2023-01-24T22:08:32.343 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_op_history_size": "20", 2023-01-24T22:08:32.344 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_op_log_threshold": "5", 2023-01-24T22:08:32.344 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_ping_grace": "15", 2023-01-24T22:08:32.344 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_ping_interval": "5", 2023-01-24T22:08:32.344 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-24T22:08:32.344 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-24T22:08:32.345 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_max_caps": "30000", 2023-01-24T22:08:32.345 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-24T22:08:32.345 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-24T22:08:32.345 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-24T22:08:32.345 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_recall_warning_threshold": "262144", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_replay_interval": "1.000000", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_root_ino_gid": "0", 2023-01-24T22:08:32.346 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_root_ino_uid": "0", 2023-01-24T22:08:32.347 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-24T22:08:32.347 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-24T22:08:32.347 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-24T22:08:32.347 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-24T22:08:32.347 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-24T22:08:32.348 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-24T22:08:32.348 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-24T22:08:32.348 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-24T22:08:32.348 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-24T22:08:32.348 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_shutdown_check": "0", 2023-01-24T22:08:32.349 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_skip_ino": "0", 2023-01-24T22:08:32.349 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-24T22:08:32.349 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_snap_max_uid": "4294967294", 2023-01-24T22:08:32.349 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_snap_min_uid": "0", 2023-01-24T22:08:32.349 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_snap_rstat": "false", 2023-01-24T22:08:32.350 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_standby_replay_damaged": "false", 2023-01-24T22:08:32.350 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_symlink_recovery": "true", 2023-01-24T22:08:32.350 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-24T22:08:32.350 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_thrash_exports": "0", 2023-01-24T22:08:32.350 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_thrash_fragments": "0", 2023-01-24T22:08:32.351 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_tick_interval": "5.000000", 2023-01-24T22:08:32.351 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_valgrind_exit": "false", 2023-01-24T22:08:32.351 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_verify_backtrace": "1", 2023-01-24T22:08:32.351 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_verify_scatter": "false", 2023-01-24T22:08:32.351 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-24T22:08:32.352 INFO:teuthology.orchestra.run.smithi035.stdout: "mds_wipe_sessions": "false", 2023-01-24T22:08:32.352 INFO:teuthology.orchestra.run.smithi035.stdout: "mempool_debug": "false", 2023-01-24T22:08:32.352 INFO:teuthology.orchestra.run.smithi035.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-24T22:08:32.352 INFO:teuthology.orchestra.run.smithi035.stdout: "memstore_device_bytes": "1073741824", 2023-01-24T22:08:32.352 INFO:teuthology.orchestra.run.smithi035.stdout: "memstore_page_set": "false", 2023-01-24T22:08:32.353 INFO:teuthology.orchestra.run.smithi035.stdout: "memstore_page_size": "65536", 2023-01-24T22:08:32.353 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_client_bytes": "134217728", 2023-01-24T22:08:32.353 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_client_messages": "512", 2023-01-24T22:08:32.353 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-24T22:08:32.354 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-24T22:08:32.354 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-24T22:08:32.354 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-24T22:08:32.354 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_disabled_modules": "", 2023-01-24T22:08:32.354 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-24T22:08:32.355 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_max_pg_num_change": "128", 2023-01-24T22:08:32.355 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_mds_bytes": "134217728", 2023-01-24T22:08:32.355 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_mds_messages": "128", 2023-01-24T22:08:32.355 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-24T22:08:32.355 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_mon_bytes": "134217728", 2023-01-24T22:08:32.356 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_mon_messages": "128", 2023-01-24T22:08:32.356 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_osd_bytes": "536870912", 2023-01-24T22:08:32.356 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_osd_messages": "8192", 2023-01-24T22:08:32.356 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_pool": "true", 2023-01-24T22:08:32.356 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-24T22:08:32.357 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_standby_modules": "true", 2023-01-24T22:08:32.357 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_stats_period": "5", 2023-01-24T22:08:32.357 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_stats_threshold": "5", 2023-01-24T22:08:32.357 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_tick_period": "2", 2023-01-24T22:08:32.357 INFO:teuthology.orchestra.run.smithi035.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_allow_pool_delete": "true", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_allow_pool_size_one": "true", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_auth_validate_all_caps": "true", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-24T22:08:32.358 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-24T22:08:32.359 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_bytes": "104857600", 2023-01-24T22:08:32.359 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_directed_command_retry": "5", 2023-01-24T22:08:32.359 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-24T22:08:32.359 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-24T22:08:32.360 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-24T22:08:32.360 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-24T22:08:32.360 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_hunt_parallel": "3", 2023-01-24T22:08:32.360 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_log_interval": "1.000000", 2023-01-24T22:08:32.360 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-24T22:08:32.361 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_ping_interval": "10.000000", 2023-01-24T22:08:32.361 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-24T22:08:32.361 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-24T22:08:32.361 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-24T22:08:32.361 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-24T22:08:32.362 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_file_level": "debug", 2023-01-24T22:08:32.362 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_file": "true", 2023-01-24T22:08:32.362 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-24T22:08:32.362 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-24T22:08:32.362 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_journald": "false", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-24T22:08:32.363 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_compact_on_bootstrap": "false", 2023-01-24T22:08:32.364 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_compact_on_start": "false", 2023-01-24T22:08:32.364 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_compact_on_trim": "true", 2023-01-24T22:08:32.364 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-24T22:08:32.364 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-24T22:08:32.364 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-24T22:08:32.365 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_cpu_threads": "4", 2023-01-24T22:08:32.365 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_crush_min_required_version": "hammer", 2023-01-24T22:08:32.365 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_daemon_bytes": "419430400", 2023-01-24T22:08:32.365 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-24T22:08:32.365 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_data_avail_crit": "5", 2023-01-24T22:08:32.366 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_data_avail_warn": "5", 2023-01-24T22:08:32.366 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_data_size_warn": "16106127360", 2023-01-24T22:08:32.366 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-24T22:08:32.366 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-24T22:08:32.366 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_dump_json": "false", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_dump_transactions": "false", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_extra_checks": "false", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-24T22:08:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_no_require_pacific": "false", 2023-01-24T22:08:32.368 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_no_require_quincy": "false", 2023-01-24T22:08:32.368 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-24T22:08:32.368 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-24T22:08:32.368 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-24T22:08:32.368 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_down_mkfs_grace": "120", 2023-01-24T22:08:32.369 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_election_default_strategy": "3", 2023-01-24T22:08:32.369 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_election_timeout": "5.000000", 2023-01-24T22:08:32.369 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-24T22:08:32.369 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_elector_ping_divisor": "2", 2023-01-24T22:08:32.369 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_enable_op_tracker": "true", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_fake_pool_delete": "false", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_force_quorum_join": "false", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_globalid_prealloc": "10000", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_detail_to_clog": "false", 2023-01-24T22:08:32.370 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_log_update_period": "5", 2023-01-24T22:08:32.371 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_max_detail": "50", 2023-01-24T22:08:32.371 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_to_clog": "true", 2023-01-24T22:08:32.371 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_to_clog_interval": "600", 2023-01-24T22:08:32.371 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-24T22:08:32.371 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_host": "[v1:172.21.15.35:6789],[v1:172.21.15.35:6790],[v1:172.21.15.72:6789]", 2023-01-24T22:08:32.372 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_host_override": "", 2023-01-24T22:08:32.372 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_initial_members": "", 2023-01-24T22:08:32.372 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-24T22:08:32.372 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-24T22:08:32.372 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-24T22:08:32.373 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-24T22:08:32.373 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-24T22:08:32.373 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_lease": "5.000000", 2023-01-24T22:08:32.373 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-24T22:08:32.373 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-24T22:08:32.374 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_log_full_interval": "50", 2023-01-24T22:08:32.374 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_log_max": "10000", 2023-01-24T22:08:32.374 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_log_max_summary": "50", 2023-01-24T22:08:32.374 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-24T22:08:32.374 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_log_epochs": "500", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_osd": "10000", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_pg_per_osd": "10000", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_pool_pg_num": "65536", 2023-01-24T22:08:32.375 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-24T22:08:32.376 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-24T22:08:32.376 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mds_force_trim_to": "0", 2023-01-24T22:08:32.376 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mds_skip_sanity": "false", 2023-01-24T22:08:32.376 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_memory_autotune": "true", 2023-01-24T22:08:32.376 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_memory_target": "2147483648", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_beacon_grace": "30", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_digest_period": "5", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_inactive_grace": "60", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-24T22:08:32.377 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-24T22:08:32.378 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_min_osdmap_epochs": "500", 2023-01-24T22:08:32.378 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_complaint_time": "30", 2023-01-24T22:08:32.378 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_history_duration": "600", 2023-01-24T22:08:32.378 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_history_size": "20", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_history_slow_op_size": "20", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_op_log_threshold": "5", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-24T22:08:32.379 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-24T22:08:32.380 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_auto_mark_in": "false", 2023-01-24T22:08:32.380 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-24T22:08:32.380 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-24T22:08:32.380 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-24T22:08:32.380 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_cache_size": "500", 2023-01-24T22:08:32.381 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-24T22:08:32.381 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-24T22:08:32.381 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-24T22:08:32.381 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_down_out_interval": "600", 2023-01-24T22:08:32.381 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-24T22:08:32.382 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-24T22:08:32.382 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_force_trim_to": "0", 2023-01-24T22:08:32.382 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-24T22:08:32.382 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-24T22:08:32.382 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-24T22:08:32.383 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_min_down_reporters": "2", 2023-01-24T22:08:32.384 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-24T22:08:32.384 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-24T22:08:32.384 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-24T22:08:32.384 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-24T22:08:32.384 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_report_timeout": "900", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-24T22:08:32.385 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-24T22:08:32.386 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-24T22:08:32.386 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osdmap_full_prune_interval": "10", 2023-01-24T22:08:32.386 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osdmap_full_prune_min": "10000", 2023-01-24T22:08:32.386 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_osdmap_full_prune_txsize": "100", 2023-01-24T22:08:32.386 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-24T22:08:32.387 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_stuck_threshold": "60", 2023-01-24T22:08:32.387 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-24T22:08:32.387 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-24T22:08:32.387 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-24T22:08:32.387 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-24T22:08:32.388 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-24T22:08:32.388 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-24T22:08:32.388 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_probe_timeout": "2.000000", 2023-01-24T22:08:32.388 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_reweight_max_change": "0.050000", 2023-01-24T22:08:32.388 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_reweight_max_osds": "4", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-24T22:08:32.389 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_scrub_interval": "300", 2023-01-24T22:08:32.390 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_scrub_max_keys": "100", 2023-01-24T22:08:32.390 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_scrub_timeout": "300", 2023-01-24T22:08:32.390 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_session_timeout": "300", 2023-01-24T22:08:32.390 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_smart_report_timeout": "5", 2023-01-24T22:08:32.390 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_stat_smooth_intervals": "6", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_stretch_pool_min_size": "2", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_stretch_pool_size": "4", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-24T22:08:32.391 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_debug": "false", 2023-01-24T22:08:32.392 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-24T22:08:32.392 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-24T22:08:32.392 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_provider_kill_at": "0", 2023-01-24T22:08:32.392 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_requester_kill_at": "0", 2023-01-24T22:08:32.392 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_sync_timeout": "60.000000", 2023-01-24T22:08:32.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_target_pg_per_osd": "100", 2023-01-24T22:08:32.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_tick_interval": "5", 2023-01-24T22:08:32.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_timecheck_interval": "300.000000", 2023-01-24T22:08:32.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-24T22:08:32.393 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_older_version_delay": "604800", 2023-01-24T22:08:32.394 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-24T22:08:32.394 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-24T22:08:32.394 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-24T22:08:32.394 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-24T22:08:32.394 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_misplaced": "false", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_older_version": "true", 2023-01-24T22:08:32.395 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-24T22:08:32.396 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-24T22:08:32.396 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-24T22:08:32.396 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-24T22:08:32.396 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-24T22:08:32.396 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-24T22:08:32.397 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-24T22:08:32.397 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-24T22:08:32.397 INFO:teuthology.orchestra.run.smithi035.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-24T22:08:32.397 INFO:teuthology.orchestra.run.smithi035.stdout: "monmap": "", 2023-01-24T22:08:32.397 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_op_threads": "3", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_cm": "false", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_device_name": "", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_dscp": "96", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-24T22:08:32.398 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-24T22:08:32.399 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_local_gid": "", 2023-01-24T22:08:32.399 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-24T22:08:32.399 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_port_num": "1", 2023-01-24T22:08:32.399 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-24T22:08:32.399 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-24T22:08:32.400 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_roce_ver": "1", 2023-01-24T22:08:32.400 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_send_buffers": "1024", 2023-01-24T22:08:32.400 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_sl": "3", 2023-01-24T22:08:32.400 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_support_srq": "true", 2023-01-24T22:08:32.400 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_rdma_type": "ib", 2023-01-24T22:08:32.401 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_async_reap_threshold": "5", 2023-01-24T22:08:32.401 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_before_connect": "false", 2023-01-24T22:08:32.401 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_ipv4": "true", 2023-01-24T22:08:32.401 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_ipv6": "false", 2023-01-24T22:08:32.401 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_msgr1": "true", 2023-01-24T22:08:32.402 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_msgr2": "false", 2023-01-24T22:08:32.402 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_port_max": "7568", 2023-01-24T22:08:32.402 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_port_min": "6800", 2023-01-24T22:08:32.402 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_prefer_ipv4": "false", 2023-01-24T22:08:32.402 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_retry_count": "3", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_bind_retry_delay": "5", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_blackhole_client": "false", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_blackhole_mds": "false", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_blackhole_mgr": "false", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_blackhole_mon": "false", 2023-01-24T22:08:32.403 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_blackhole_osd": "false", 2023-01-24T22:08:32.404 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_client_mode": "crc secure", 2023-01-24T22:08:32.404 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_cluster_mode": "crc secure", 2023-01-24T22:08:32.404 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_cluster_type": "", 2023-01-24T22:08:32.404 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_compress_secure": "false", 2023-01-24T22:08:32.404 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_connection_idle_timeout": "900", 2023-01-24T22:08:32.405 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_connection_ready_timeout": "10", 2023-01-24T22:08:32.405 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_crc_data": "true", 2023-01-24T22:08:32.405 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_crc_header": "true", 2023-01-24T22:08:32.405 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_die_on_bad_msg": "false", 2023-01-24T22:08:32.405 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_die_on_bug": "true", 2023-01-24T22:08:32.406 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_die_on_old_message": "true", 2023-01-24T22:08:32.406 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_die_on_skipped_message": "false", 2023-01-24T22:08:32.406 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_die_on_unhandled_msg": "false", 2023-01-24T22:08:32.406 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-01-24T22:08:32.406 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_coremask": "0xF", 2023-01-24T22:08:32.407 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-01-24T22:08:32.407 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_devs_allowlist": "", 2023-01-24T22:08:32.407 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_enable_tso": "true", 2023-01-24T22:08:32.407 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-01-24T22:08:32.407 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-01-24T22:08:32.408 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_hugepages": "", 2023-01-24T22:08:32.408 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_hw_flow_control": "true", 2023-01-24T22:08:32.408 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-01-24T22:08:32.408 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_lro": "true", 2023-01-24T22:08:32.408 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_memory_channel": "4", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_pmd": "", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_port_id": "0", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dump_corrupt_message_level": "1", 2023-01-24T22:08:32.409 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_dump_on_send": "false", 2023-01-24T22:08:32.410 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_initial_backoff": "0.200000", 2023-01-24T22:08:32.410 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_inject_delay_max": "1.000000", 2023-01-24T22:08:32.410 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_inject_delay_probability": "0.000000", 2023-01-24T22:08:32.410 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_inject_delay_type": "", 2023-01-24T22:08:32.410 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_inject_internal_delays": "0.000000", 2023-01-24T22:08:32.411 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_inject_socket_failures": "2500", 2023-01-24T22:08:32.411 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_learn_addr_from_peer": "true", 2023-01-24T22:08:32.411 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_max_accept_failures": "4", 2023-01-24T22:08:32.411 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_max_backoff": "15.000000", 2023-01-24T22:08:32.411 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_mon_client_mode": "secure crc", 2023-01-24T22:08:32.412 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_mon_cluster_mode": "secure crc", 2023-01-24T22:08:32.412 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_mon_service_mode": "secure crc", 2023-01-24T22:08:32.412 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_osd_compress_min_size": "1024", 2023-01-24T22:08:32.412 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_osd_compress_mode": "none", 2023-01-24T22:08:32.412 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_pq_min_cost": "65536", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_public_type": "", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_service_mode": "crc secure", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_tcp_listen_backlog": "512", 2023-01-24T22:08:32.413 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_tcp_nodelay": "true", 2023-01-24T22:08:32.414 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-01-24T22:08:32.414 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_tcp_rcvbuf": "0", 2023-01-24T22:08:32.414 INFO:teuthology.orchestra.run.smithi035.stdout: "ms_type": "async", 2023-01-24T22:08:32.414 INFO:teuthology.orchestra.run.smithi035.stdout: "no_config_file": "false", 2023-01-24T22:08:32.414 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_completion_locks_per_session": "32", 2023-01-24T22:08:32.415 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_debug_inject_relock_delay": "false", 2023-01-24T22:08:32.415 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_inflight_op_bytes": "104857600", 2023-01-24T22:08:32.415 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_inflight_ops": "1024", 2023-01-24T22:08:32.415 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_inject_no_watch_ping": "false", 2023-01-24T22:08:32.415 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_tick_interval": "5.000000", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "objecter_timeout": "10.000000", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "objectstore_blackhole": "false", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "openssl_engine_opts": "", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_delay_time": "5.000000", 2023-01-24T22:08:32.416 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_hist_halflife": "1000", 2023-01-24T22:08:32.417 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_max_low_ops": "2", 2023-01-24T22:08:32.417 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_max_ops": "4", 2023-01-24T22:08:32.417 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-01-24T22:08:32.417 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_quantize_effort": "0.100000", 2023-01-24T22:08:32.417 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_agent_slop": "0.020000", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_allow_recovery_below_min_size": "true", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_asio_thread_count": "2", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_async_recovery_min_cost": "100", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_auto_mark_unfound_lost": "false", 2023-01-24T22:08:32.418 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backfill_retry_interval": "30.000000", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backfill_scan_max": "512", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backfill_scan_min": "64", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backoff_on_degraded": "false", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backoff_on_peering": "false", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_backoff_on_unfound": "true", 2023-01-24T22:08:32.419 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_beacon_report_interval": "300", 2023-01-24T22:08:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_bench_duration": "30", 2023-01-24T22:08:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-01-24T22:08:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_bench_max_block_size": "67108864", 2023-01-24T22:08:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_bench_small_size_max_iops": "100", 2023-01-24T22:08:32.420 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_blkin_trace_all": "false", 2023-01-24T22:08:32.421 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-01-24T22:08:32.421 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-01-24T22:08:32.421 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_check_for_log_corruption": "false", 2023-01-24T22:08:32.421 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-01-24T22:08:32.422 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T22:08:32.422 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-01-24T22:08:32.422 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T22:08:32.422 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_class_update_on_start": "true", 2023-01-24T22:08:32.422 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_client_message_cap": "256", 2023-01-24T22:08:32.423 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_client_message_size_cap": "524288000", 2023-01-24T22:08:32.423 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_client_op_priority": "63", 2023-01-24T22:08:32.423 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_client_watch_timeout": "30", 2023-01-24T22:08:32.423 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_command_max_records": "256", 2023-01-24T22:08:32.423 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_compact_on_start": "false", 2023-01-24T22:08:32.424 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-01-24T22:08:32.424 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_crush_chooseleaf_type": "0", 2023-01-24T22:08:32.424 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_crush_initial_weight": "-1.000000", 2023-01-24T22:08:32.424 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_crush_update_on_start": "true", 2023-01-24T22:08:32.424 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_crush_update_weight_set": "true", 2023-01-24T22:08:32.425 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-01-24T22:08:32.425 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-01-24T22:08:32.425 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-01-24T22:08:32.425 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_drop_ping_duration": "0", 2023-01-24T22:08:32.425 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-01-24T22:08:32.426 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_feed_pullee": "-1", 2023-01-24T22:08:32.426 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-01-24T22:08:32.426 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-01-24T22:08:32.426 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-01-24T22:08:32.426 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_misdirected_ops": "false", 2023-01-24T22:08:32.427 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_no_acting_change": "false", 2023-01-24T22:08:32.427 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_no_purge_strays": "false", 2023-01-24T22:08:32.427 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_op_order": "false", 2023-01-24T22:08:32.427 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_pg_log_writeout": "false", 2023-01-24T22:08:32.428 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_pretend_recovery_active": "false", 2023-01-24T22:08:32.428 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-01-24T22:08:32.428 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-01-24T22:08:32.428 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_shutdown": "false", 2023-01-24T22:08:32.428 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-01-24T22:08:32.429 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-01-24T22:08:32.429 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_verify_cached_snaps": "false", 2023-01-24T22:08:32.429 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_verify_missing_on_start": "false", 2023-01-24T22:08:32.429 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_verify_snaps": "false", 2023-01-24T22:08:32.429 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-01-24T22:08:32.430 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-01-24T22:08:32.430 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_keys": "1024", 2023-01-24T22:08:32.430 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-01-24T22:08:32.430 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-01-24T22:08:32.430 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-01-24T22:08:32.431 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_stride": "524288", 2023-01-24T22:08:32.431 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-01-24T22:08:32.431 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_default_data_pool_replay_window": "5", 2023-01-24T22:08:32.431 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_default_notify_timeout": "30", 2023-01-24T22:08:32.432 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_delete_sleep": "0.000000", 2023-01-24T22:08:32.432 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-01-24T22:08:32.432 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-01-24T22:08:32.432 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-01-24T22:08:32.432 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_discard_disconnected_ops": "true", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_enable_op_tracker": "true", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_fast_info": "true", 2023-01-24T22:08:32.433 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_fast_shutdown": "true", 2023-01-24T22:08:32.434 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-01-24T22:08:32.434 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_fast_shutdown_timeout": "15", 2023-01-24T22:08:32.434 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-01-24T22:08:32.434 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-01-24T22:08:32.434 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-01-24T22:08:32.435 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_function_tracing": "false", 2023-01-24T22:08:32.435 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_grace": "20", 2023-01-24T22:08:32.435 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_interval": "6", 2023-01-24T22:08:32.435 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-01-24T22:08:32.435 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_min_peers": "10", 2023-01-24T22:08:32.436 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_min_size": "2000", 2023-01-24T22:08:32.436 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_stale": "600", 2023-01-24T22:08:32.436 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-01-24T22:08:32.436 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_hit_set_max_size": "100000", 2023-01-24T22:08:32.436 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_hit_set_min_size": "1000", 2023-01-24T22:08:32.437 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-01-24T22:08:32.437 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-01-24T22:08:32.437 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-01-24T22:08:32.437 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-01-24T22:08:32.437 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_journal_flush_on_shutdown": "true", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_journal_size": "5120", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_kill_backfill_at": "0", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_loop_before_reset_tphandle": "64", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_map_cache_size": "50", 2023-01-24T22:08:32.438 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_map_dedup": "true", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_map_message_max": "40", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_map_message_max_bytes": "10485760", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_map_share_max_epochs": "40", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_attr_name_len": "100", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_attr_size": "0", 2023-01-24T22:08:32.439 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_backfills": "1", 2023-01-24T22:08:32.440 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_markdown_count": "5", 2023-01-24T22:08:32.440 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_markdown_period": "600", 2023-01-24T22:08:32.440 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_object_name_len": "2048", 2023-01-24T22:08:32.440 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_object_namespace_len": "256", 2023-01-24T22:08:32.440 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_object_size": "134217728", 2023-01-24T22:08:32.441 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-01-24T22:08:32.441 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_omap_entries_per_request": "1024", 2023-01-24T22:08:32.441 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_pg_blocked_by": "16", 2023-01-24T22:08:32.441 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_pg_log_entries": "2", 2023-01-24T22:08:32.441 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-01-24T22:08:32.442 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_pgls": "1024", 2023-01-24T22:08:32.442 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_push_cost": "8388608", 2023-01-24T22:08:32.442 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_push_objects": "10", 2023-01-24T22:08:32.442 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_scrubs": "1", 2023-01-24T22:08:32.442 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-01-24T22:08:32.443 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_trimming_pgs": "2", 2023-01-24T22:08:32.443 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_write_op_reply_len": "64", 2023-01-24T22:08:32.443 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_max_write_size": "90", 2023-01-24T22:08:32.443 INFO:teuthology.orchestra.run.smithi035.stdout: "osd_mclock_cost_per