2023-01-24T18:21:11.206 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T18:21:11.207 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589 -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589/orig.config.yaml 2023-01-24T18:21:11.259 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T18:21:11.331 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} email: yweinstein@radhat.com first_in_suite: false job_id: '7135589' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on peering: true osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Qg8vdWmitOAr1GSJyROpi3BklFzbLUQDd9fO0REOvEhywBNzNewiDi9iMmyIOKnz6nW2LapuKifLXw4MchFEi+bVfSKZ2PhHlylj2qOj5CBAzxGR+2yNrTmEveoWayZuqKqIoLQ1CpAtPIIKlOfCXs+ycP+cz3NhICtgswpuhOJ9nNMKg4aD+1IT0jeRAW/FVvNbPngrbXxF1uzfMFTWoIw0ylhGItPCZBY2O6iKMKE15tdA/q5sU6VHxAYNZvLhzYNFxhBfT55Ux4vBqFW32FLFlmAa0qP48dN0hIpACDJEai0ZlxoIpIKhocq6xZqhT3GUAKXmhMm5djV6VrtjUvytNMYbkMr2P3VkD7aRwVymhF+TGokcckaBWoE5MmF0LtHYBLHD6W1MaxgV8SVwfAFey6B56jZGRii6gGkkEZVT0PzxRnGAX6Ik5o6vl4lUaNrzyv/+BjFmWNFSkEEgdZdILlC7xYxxoyIYh8kI5YFC5vL3cEVRwHLqvE/nIS8= smithi186.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxp/U1GuW4XESvFD2xHUvyhjJcR+j5BbtoSayqe23c/pGRx0Bnuj83ynTzvhfbsbW8vVoBizOMp9BaSTRSHUSi+VCFTEjNk/qsUNQlPIXDS+hgkbzVFKxT7T1HB7TLRUIFTRJIF0JNCqyMLf+F/2POxhUBwV5ksWSJD4CqownVJJnhNcn2xwu+7OwDR0SOlEVnatjnhNTiUyCC54yFfYqQl/QTBjEvQV0CyFAecAbD1YjPmphNH690rHVVIykDpVE0MUgIfDm4JJ9jIJLqZKDZrf7+jn39vfc5bdbxT2pd/4euRkDsJBycy0q+QupqUdZtlMXbeA686dMjo0gv+/erHJ7XK3jSA434N3iS7bDJUT7i4VrxWty4UuQIPjs7YgNGygwDe0aaNjjUchAs+ivHBr5fzNGeOgNUNPybgDjgBXIMekwh1sfWhF31M5+Zx1cqcE5jXIuDOSLKWvGFUePtXl6zWvn/B8Ve58G+aVspWXhGANgQqHnwT9D9N1z+3/E= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache readproxy - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T18:21:11.332 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T18:21:11.333 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T18:21:11.334 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T18:21:11.336 INFO:teuthology.task.internal:Checking packages... 2023-01-24T18:21:11.383 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T18:21:11.384 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T18:21:11.385 INFO:teuthology.packaging:ref: None 2023-01-24T18:21:11.385 INFO:teuthology.packaging:tag: None 2023-01-24T18:21:11.386 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:21:11.386 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:21:11.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:21:11.543 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T18:21:11.543 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T18:21:11.551 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T18:21:11.551 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T18:21:11.574 INFO:teuthology.task.internal:Saving configuration 2023-01-24T18:21:11.593 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T18:21:11.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T18:21:11.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:15:57.936583', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZzDDaACfya/Acju9klEEtbqLmCIOcMPr0W46OMoBMOY/jda3UY4KgpTigCf7kjGe0Y9U5LBStDXLkQPVSTgDs='} 2023-01-24T18:21:11.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:15:57.935153', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHH7DzeApjJFVZIGqwLYm12W2aaoxu7y3jNngGfPKAUkd/XqU80O0ZwpACvRPZ1r9PEKuF/t6E5f+WnSOkaWo/0='} 2023-01-24T18:21:11.643 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T18:21:11.650 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T18:21:11.650 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T18:21:11.651 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T18:21:11.724 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f96d5862670>, signals=[15]) 2023-01-24T18:21:11.725 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T18:21:11.732 INFO:teuthology.task.internal:Opening connections... 2023-01-24T18:21:11.732 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:21:11.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:21:11.803 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:21:11.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:21:11.870 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T18:21:11.877 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-01-24T18:21:11.895 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-01-24T18:21:11.896 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-01-24T18:21:11.952 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T18:21:11.952 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T18:21:11.952 INFO:teuthology.orchestra.run.smithi169.stdout:ID="rhel" 2023-01-24T18:21:11.953 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="fedora" 2023-01-24T18:21:11.953 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8.4" 2023-01-24T18:21:11.953 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8" 2023-01-24T18:21:11.953 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T18:21:11.953 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2023-01-24T18:21:11.954 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T18:21:11.954 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T18:21:11.954 INFO:teuthology.orchestra.run.smithi169.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T18:21:11.954 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T18:21:11.955 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:21:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T18:21:11.955 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T18:21:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T18:21:11.956 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T18:21:11.956 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-01-24T18:21:11.973 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-01-24T18:21:11.991 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-01-24T18:21:11.992 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-01-24T18:21:12.047 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T18:21:12.047 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T18:21:12.047 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-01-24T18:21:12.047 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-01-24T18:21:12.048 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.4" 2023-01-24T18:21:12.048 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-01-24T18:21:12.048 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T18:21:12.048 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-01-24T18:21:12.048 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T18:21:12.049 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T18:21:12.049 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T18:21:12.049 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T18:21:12.049 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:21:12.049 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T18:21:12.050 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T18:21:12.050 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T18:21:12.050 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T18:21:12.050 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-01-24T18:21:12.065 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T18:21:12.075 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T18:21:12.081 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T18:21:12.082 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2023-01-24T18:21:12.085 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-01-24T18:21:12.102 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T18:21:12.109 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T18:21:12.110 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2023-01-24T18:21:12.143 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-01-24T18:21:12.155 INFO:teuthology.orchestra.run.smithi169.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T18:21:12.159 INFO:teuthology.orchestra.run.smithi186.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T18:21:12.160 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T18:21:12.209 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T18:21:12.226 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T18:21:12.226 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T18:21:12.227 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T18:21:12.228 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T18:21:12.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:21:12.243 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-01-24T18:21:12.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:21:12.245 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-01-24T18:21:12.299 INFO:teuthology.orchestra.run.smithi169.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:12.300 INFO:teuthology.task.kernel:Running kernel on smithi169: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:12.300 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum install -y kernel 2023-01-24T18:21:12.301 INFO:teuthology.orchestra.run.smithi186.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:12.301 INFO:teuthology.task.kernel:Running kernel on smithi186: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:12.302 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum install -y kernel 2023-01-24T18:21:13.717 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:13.718 INFO:teuthology.orchestra.run.smithi169.stdout:Unable to read consumer identity 2023-01-24T18:21:13.759 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:13.760 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-01-24T18:21:15.089 INFO:teuthology.orchestra.run.smithi169.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-01-24T18:21:15.095 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 32 kB/s | 3.5 kB 00:00 2023-01-24T18:21:15.918 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 17 MB/s | 13 MB 00:00 2023-01-24T18:21:15.938 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 16 MB/s | 13 MB 00:00 2023-01-24T18:21:18.719 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-01-24T18:21:18.720 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-01-24T18:21:19.425 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T18:21:19.426 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T18:21:19.426 INFO:teuthology.orchestra.run.smithi186.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T18:21:19.430 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T18:21:19.430 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T18:21:19.430 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T18:21:19.560 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:21:19.561 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:21:19.561 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:21:19.576 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:21:19.577 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:21:19.577 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:21:19.677 DEBUG:teuthology.orchestra.run.smithi186:> echo no | sudo yum reinstall kernel || true 2023-01-24T18:21:19.705 DEBUG:teuthology.orchestra.run.smithi169:> echo no | sudo yum reinstall kernel || true 2023-01-24T18:21:20.064 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:20.064 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-01-24T18:21:20.094 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:20.094 INFO:teuthology.orchestra.run.smithi169.stdout:Unable to read consumer identity 2023-01-24T18:21:20.348 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 06:21:18 PM UTC. 2023-01-24T18:21:20.385 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 06:21:18 PM UTC. 2023-01-24T18:21:20.485 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:21:20.485 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:20.486 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:20.511 INFO:teuthology.orchestra.run.smithi186.stderr:Error: No packages marked for reinstall. 2023-01-24T18:21:20.548 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:21:20.548 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:20.549 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:20.562 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum reinstall -y kernel || true 2023-01-24T18:21:20.591 INFO:teuthology.orchestra.run.smithi169.stderr:Error: No packages marked for reinstall. 2023-01-24T18:21:20.641 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum reinstall -y kernel || true 2023-01-24T18:21:20.906 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:20.906 INFO:teuthology.orchestra.run.smithi186.stdout:Unable to read consumer identity 2023-01-24T18:21:20.986 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:21:20.986 INFO:teuthology.orchestra.run.smithi169.stdout:Unable to read consumer identity 2023-01-24T18:21:21.190 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:03 ago on Tue 24 Jan 2023 06:21:18 PM UTC. 2023-01-24T18:21:21.266 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:03 ago on Tue 24 Jan 2023 06:21:18 PM UTC. 2023-01-24T18:21:21.334 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:21:21.335 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:21.335 INFO:teuthology.orchestra.run.smithi186.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:21.353 INFO:teuthology.orchestra.run.smithi186.stderr:Error: No packages marked for reinstall. 2023-01-24T18:21:21.406 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q kernel --last | head -n 1 2023-01-24T18:21:21.411 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:21:21.411 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:21.411 INFO:teuthology.orchestra.run.smithi169.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:21:21.420 INFO:teuthology.orchestra.run.smithi169.stderr:Error: No packages marked for reinstall. 2023-01-24T18:21:21.470 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel --last | head -n 1 2023-01-24T18:21:21.531 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T18:21:21.532 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:21.532 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T18:21:21.532 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T18:21:21.533 DEBUG:teuthology.orchestra.run.smithi186:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T18:21:21.567 INFO:teuthology.orchestra.run.smithi186.stdout:ttyS1 2023-01-24T18:21:21.583 DEBUG:teuthology.parallel:result is None 2023-01-24T18:21:21.593 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T18:21:21.595 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:21:21.595 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T18:21:21.596 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T18:21:21.596 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T18:21:21.630 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2023-01-24T18:21:21.643 DEBUG:teuthology.parallel:result is None 2023-01-24T18:21:21.643 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T18:21:21.652 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T18:21:21.653 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T18:21:21.685 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T18:21:21.699 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T18:21:21.707 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T18:21:21.719 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T18:21:21.719 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T18:21:21.739 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T18:21:21.775 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T18:21:21.782 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T18:21:21.783 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T18:21:21.806 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T18:21:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:21:21.837 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:21:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:21:21.847 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:21:21.848 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T18:21:21.856 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T18:21:21.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T18:21:21.886 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T18:21:21.912 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T18:21:21.921 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T18:21:21.921 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T18:21:21.950 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T18:21:21.969 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:21:22.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:21:22.082 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:21:22.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:21:22.202 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:21:22.202 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T18:21:22.265 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:21:22.313 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:21:22.348 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:21:22.435 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:21:22.467 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:21:22.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T18:21:22.530 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2023-01-24T18:21:22.533 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-01-24T18:21:22.574 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T18:21:22.624 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T18:21:22.889 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T18:21:22.897 INFO:teuthology.task.internal:Starting timer... 2023-01-24T18:21:22.898 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T18:21:22.907 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T18:21:22.918 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T18:21:22.918 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2023-01-24T18:21:22.978 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: [ OK ] 2023-01-24T18:21:22.979 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-01-24T18:21:23.037 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: [ OK ] 2023-01-24T18:21:23.039 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T18:21:23.039 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2023-01-24T18:21:23.065 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2023-01-24T18:21:23.065 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-01-24T18:21:23.086 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-01-24T18:21:23.087 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi169.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-01-24T18:21:23.087 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T18:21:23.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:21:23.129 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T18:21:23.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:21:23.155 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T18:21:23.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2023-01-24T18:21:23.203 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-01-24T18:21:23.229 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T18:21:23.245 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-24T18:21:23.247 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T18:21:23.265 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T18:21:23.266 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi169.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T18:27:49.682 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi169.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-01-24T18:27:49.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-01-24T18:27:49.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:27:49.754 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-01-24T18:27:49.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-01-24T18:27:49.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:27:49.886 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-01-24T18:27:49.955 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T18:27:49.966 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T18:27:49.967 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T18:27:49.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T18:27:49.970 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T18:27:49.970 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T18:27:49.997 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T18:27:50.010 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T18:27:50.025 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T18:27:50.040 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T18:27:50.044 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2023-01-24T18:27:50.054 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2023-01-24T18:27:50.067 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T18:27:50.074 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-01-24T18:27:50.080 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T18:27:50.084 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-01-24T18:27:50.098 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T18:27:50.111 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T18:27:50.139 INFO:teuthology.orchestra.run.smithi169.stderr:bash: ntpq: command not found 2023-01-24T18:27:50.142 INFO:teuthology.orchestra.run.smithi169.stdout:210 Number of sources = 4 2023-01-24T18:27:50.142 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T18:27:50.142 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2023-01-24T18:27:50.142 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.143 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.143 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.143 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.179 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-01-24T18:27:50.182 INFO:teuthology.orchestra.run.smithi186.stdout:210 Number of sources = 4 2023-01-24T18:27:50.182 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T18:27:50.182 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-01-24T18:27:50.182 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.183 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.183 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.183 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:27:50.183 INFO:teuthology.run_tasks:Running task install... 2023-01-24T18:27:50.192 DEBUG:teuthology.task.install:project ceph 2023-01-24T18:27:50.193 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T18:27:50.193 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T18:27:50.193 INFO:teuthology.task.install:Using flavor: default 2023-01-24T18:27:50.199 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T18:27:50.200 INFO:teuthology.task.install:extra packages: [] 2023-01-24T18:27:50.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo lsb_release -is 2023-01-24T18:27:50.239 INFO:teuthology.orchestra.run.smithi169.stdout:RedHatEnterprise 2023-01-24T18:27:50.240 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:27:50.240 DEBUG:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2023-01-24T18:27:50.241 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T18:27:50.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo lsb_release -is 2023-01-24T18:27:50.278 INFO:teuthology.orchestra.run.smithi186.stdout:RedHatEnterprise 2023-01-24T18:27:50.279 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:27:50.280 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T18:27:50.280 DEBUG:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2023-01-24T18:27:50.316 INFO:teuthology.orchestra.run.smithi169.stdout:RedHatEnterprise 2023-01-24T18:27:50.317 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:27:50.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:27:50.357 INFO:teuthology.orchestra.run.smithi186.stdout:RedHatEnterprise 2023-01-24T18:27:50.359 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:27:50.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:27:50.490 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T18:27:50.490 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T18:27:50.516 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T18:27:50.517 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T18:27:50.612 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T18:27:50.613 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:27:50.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T18:27:50.636 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T18:27:50.637 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:27:50.637 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T18:27:50.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T18:27:50.639 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T18:27:50.665 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T18:27:50.665 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T18:27:50.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T18:27:50.728 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T18:27:50.776 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T18:27:50.801 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T18:27:50.838 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2023-01-24T18:27:50.840 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2023-01-24T18:27:50.864 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-01-24T18:27:50.865 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-01-24T18:27:51.224 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:27:51.252 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:27:51.554 INFO:teuthology.orchestra.run.smithi169.stdout:44 files removed 2023-01-24T18:27:51.563 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-01-24T18:27:51.604 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw 2023-01-24T18:27:51.613 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw 2023-01-24T18:27:51.948 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:27:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:27:52.652 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 317 kB/s | 78 kB 00:00 2023-01-24T18:27:52.668 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 319 kB/s | 78 kB 00:00 2023-01-24T18:27:52.949 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 36 kB/s | 9.2 kB 00:00 2023-01-24T18:27:52.968 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 36 kB/s | 9.2 kB 00:00 2023-01-24T18:27:53.349 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 3.9 kB/s | 1.5 kB 00:00 2023-01-24T18:27:53.379 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-01-24T18:27:53.464 INFO:teuthology.orchestra.run.smithi169.stdout:Copr repo for python3-asyncssh owned by ceph 40 kB/s | 3.5 kB 00:00 2023-01-24T18:27:53.487 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2023-01-24T18:27:55.358 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 13 MB 00:01 2023-01-24T18:27:55.893 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 5.6 MB/s | 13 MB 00:02 2023-01-24T18:27:58.330 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-01-24T18:27:58.706 INFO:teuthology.orchestra.run.smithi169.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-01-24T18:27:58.754 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-01-24T18:27:59.127 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-01-24T18:28:04.437 INFO:teuthology.orchestra.run.smithi169.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 125 MB/s | 33 MB 00:00 2023-01-24T18:28:04.709 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 124 MB/s | 33 MB 00:00 2023-01-24T18:28:13.158 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:28:13.161 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================== 2023-01-24T18:28:13.161 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:28:13.161 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================== 2023-01-24T18:28:13.161 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:28:13.161 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:28:13.162 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-01-24T18:28:13.162 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:28:13.162 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T18:28:13.162 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:28:13.163 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T18:28:13.163 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T18:28:13.163 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T18:28:13.163 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T18:28:13.163 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T18:28:13.164 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T18:28:13.164 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T18:28:13.164 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T18:28:13.164 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T18:28:13.164 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T18:28:13.165 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T18:28:13.165 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T18:28:13.165 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T18:28:13.165 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T18:28:13.166 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T18:28:13.166 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T18:28:13.166 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T18:28:13.166 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T18:28:13.166 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T18:28:13.167 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T18:28:13.167 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:13.167 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:28:13.167 INFO:teuthology.orchestra.run.smithi186.stdout:=================================================================================================== 2023-01-24T18:28:13.167 INFO:teuthology.orchestra.run.smithi186.stdout:Install 21 Packages 2023-01-24T18:28:13.168 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 2 Packages 2023-01-24T18:28:13.168 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:13.168 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 63 M 2023-01-24T18:28:13.169 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:28:13.344 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:28:13.347 INFO:teuthology.orchestra.run.smithi169.stdout:=================================================================================================== 2023-01-24T18:28:13.347 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:28:13.348 INFO:teuthology.orchestra.run.smithi169.stdout:=================================================================================================== 2023-01-24T18:28:13.348 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:28:13.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:28:13.348 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2023-01-24T18:28:13.349 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:28:13.349 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T18:28:13.349 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:28:13.349 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T18:28:13.349 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T18:28:13.350 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T18:28:13.350 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T18:28:13.350 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T18:28:13.351 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T18:28:13.351 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T18:28:13.351 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T18:28:13.351 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T18:28:13.351 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T18:28:13.352 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T18:28:13.352 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T18:28:13.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T18:28:13.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T18:28:13.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T18:28:13.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T18:28:13.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T18:28:13.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T18:28:13.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T18:28:13.354 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T18:28:13.354 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:13.354 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:28:13.354 INFO:teuthology.orchestra.run.smithi169.stdout:=================================================================================================== 2023-01-24T18:28:13.355 INFO:teuthology.orchestra.run.smithi169.stdout:Install 21 Packages 2023-01-24T18:28:13.355 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 2 Packages 2023-01-24T18:28:13.355 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:13.356 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 63 M 2023-01-24T18:28:13.356 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:28:13.869 INFO:teuthology.orchestra.run.smithi186.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 7.3 MB/s | 5.1 MB 00:00 2023-01-24T18:28:13.916 INFO:teuthology.orchestra.run.smithi186.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 515 kB/s | 24 kB 00:00 2023-01-24T18:28:14.044 INFO:teuthology.orchestra.run.smithi186.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 6.1 MB/s | 793 kB 00:00 2023-01-24T18:28:14.117 INFO:teuthology.orchestra.run.smithi186.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 6.8 MB/s | 506 kB 00:00 2023-01-24T18:28:14.885 INFO:teuthology.orchestra.run.smithi186.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 7.2 MB/s | 5.5 MB 00:00 2023-01-24T18:28:14.940 INFO:teuthology.orchestra.run.smithi186.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 818 kB/s | 45 kB 00:00 2023-01-24T18:28:14.999 INFO:teuthology.orchestra.run.smithi186.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.8 MB/s | 108 kB 00:00 2023-01-24T18:28:15.065 INFO:teuthology.orchestra.run.smithi186.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 2.8 MB/s | 190 kB 00:00 2023-01-24T18:28:15.131 INFO:teuthology.orchestra.run.smithi186.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 5.4 MB/s | 370 kB 00:00 2023-01-24T18:28:15.282 INFO:teuthology.orchestra.run.smithi186.stdout:(10/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 7.9 MB/s | 17 MB 00:02 2023-01-24T18:28:15.286 INFO:teuthology.orchestra.run.smithi186.stdout:(11/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 2.3 MB/s | 357 kB 00:00 2023-01-24T18:28:15.320 INFO:teuthology.orchestra.run.smithi186.stdout:(12/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 2.8 MB/s | 107 kB 00:00 2023-01-24T18:28:15.428 INFO:teuthology.orchestra.run.smithi186.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 545 kB/s | 59 kB 00:00 2023-01-24T18:28:15.454 INFO:teuthology.orchestra.run.smithi186.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-01-24T18:28:15.485 INFO:teuthology.orchestra.run.smithi186.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-01-24T18:28:15.522 INFO:teuthology.orchestra.run.smithi186.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.2 MB/s | 47 kB 00:00 2023-01-24T18:28:15.556 INFO:teuthology.orchestra.run.smithi169.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 2.3 MB/s | 5.1 MB 00:02 2023-01-24T18:28:15.582 INFO:teuthology.orchestra.run.smithi169.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 955 kB/s | 24 kB 00:00 2023-01-24T18:28:15.654 INFO:teuthology.orchestra.run.smithi186.stdout:(17/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 9.3 MB/s | 23 MB 00:02 2023-01-24T18:28:15.655 INFO:teuthology.orchestra.run.smithi169.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 11 MB/s | 793 kB 00:00 2023-01-24T18:28:15.703 INFO:teuthology.orchestra.run.smithi169.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 10 MB/s | 506 kB 00:00 2023-01-24T18:28:15.729 INFO:teuthology.orchestra.run.smithi186.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 212 kB/s | 44 kB 00:00 2023-01-24T18:28:15.743 INFO:teuthology.orchestra.run.smithi186.stdout:(19/23): thrift-0.13.0-2.el8.x86_64.rpm 6.0 MB/s | 1.7 MB 00:00 2023-01-24T18:28:15.751 INFO:teuthology.orchestra.run.smithi186.stdout:(20/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.6 MB/s | 259 kB 00:00 2023-01-24T18:28:15.753 INFO:teuthology.orchestra.run.smithi186.stdout:(21/23): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-01-24T18:28:16.131 INFO:teuthology.orchestra.run.smithi186.stdout:(22/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 9.5 MB/s | 3.6 MB 00:00 2023-01-24T18:28:16.163 INFO:teuthology.orchestra.run.smithi186.stdout:(23/23): librados2-17.2.5-727.g49f8fb05.el8.x86 8.2 MB/s | 3.5 MB 00:00 2023-01-24T18:28:16.166 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:28:16.166 INFO:teuthology.orchestra.run.smithi186.stdout:Total 21 MB/s | 63 MB 00:03 2023-01-24T18:28:16.170 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:28:16.214 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:28:16.214 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:28:16.236 INFO:teuthology.orchestra.run.smithi169.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 10 MB/s | 5.5 MB 00:00 2023-01-24T18:28:16.262 INFO:teuthology.orchestra.run.smithi169.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 1.7 MB/s | 45 kB 00:00 2023-01-24T18:28:16.287 INFO:teuthology.orchestra.run.smithi169.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 4.3 MB/s | 108 kB 00:00 2023-01-24T18:28:16.313 INFO:teuthology.orchestra.run.smithi169.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 7.3 MB/s | 190 kB 00:00 2023-01-24T18:28:16.341 INFO:teuthology.orchestra.run.smithi169.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 13 MB/s | 370 kB 00:00 2023-01-24T18:28:16.370 INFO:teuthology.orchestra.run.smithi169.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 12 MB/s | 357 kB 00:00 2023-01-24T18:28:16.395 INFO:teuthology.orchestra.run.smithi169.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 4.3 MB/s | 107 kB 00:00 2023-01-24T18:28:16.556 INFO:teuthology.orchestra.run.smithi169.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-01-24T18:28:16.585 INFO:teuthology.orchestra.run.smithi169.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 2.1 MB/s | 59 kB 00:00 2023-01-24T18:28:16.612 INFO:teuthology.orchestra.run.smithi169.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 2.7 MB/s | 75 kB 00:00 2023-01-24T18:28:16.807 INFO:teuthology.orchestra.run.smithi169.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 8.9 MB/s | 1.7 MB 00:00 2023-01-24T18:28:16.834 INFO:teuthology.orchestra.run.smithi169.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.7 MB/s | 47 kB 00:00 2023-01-24T18:28:16.846 INFO:teuthology.orchestra.run.smithi169.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 3.6 MB/s | 44 kB 00:00 2023-01-24T18:28:16.861 INFO:teuthology.orchestra.run.smithi169.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-01-24T18:28:16.879 INFO:teuthology.orchestra.run.smithi169.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-01-24T18:28:16.982 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:28:16.986 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:28:17.091 INFO:teuthology.orchestra.run.smithi169.stdout:(20/23): librados2-17.2.5-727.g49f8fb05.el8.x86 16 MB/s | 3.5 MB 00:00 2023-01-24T18:28:17.287 INFO:teuthology.orchestra.run.smithi169.stdout:(21/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 18 MB/s | 3.6 MB 00:00 2023-01-24T18:28:18.120 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:28:18.331 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:28:18.716 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:28:18.915 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:19.024 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:19.035 INFO:teuthology.orchestra.run.smithi169.stdout:(22/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 2.9 MB/s | 17 MB 00:05 2023-01-24T18:28:19.213 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:28:19.499 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T18:28:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:20.118 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:20.171 INFO:teuthology.orchestra.run.smithi169.stdout:(23/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 3.4 MB/s | 23 MB 00:06 2023-01-24T18:28:20.175 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:28:20.175 INFO:teuthology.orchestra.run.smithi169.stdout:Total 9.2 MB/s | 63 MB 00:06 2023-01-24T18:28:20.180 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:28:20.228 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:28:20.229 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:28:20.381 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:20.469 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:20.739 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:28:20.807 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:28:20.976 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:28:20.980 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:28:21.184 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T18:28:21.866 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T18:28:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:22.120 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:28:22.136 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:22.250 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T18:28:22.354 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:28:22.459 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:28:22.716 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:28:22.741 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:28:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T18:28:22.987 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:23.089 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:23.147 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:28:23.217 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:28:23.293 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:28:23.417 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T18:28:23.553 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T18:28:23.610 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T18:28:23.754 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:23.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T18:28:24.065 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T18:28:24.141 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:24.362 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:24.472 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:24.687 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:28:24.755 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:28:25.101 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T18:28:25.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T18:28:25.919 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:25.984 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:26.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T18:28:26.398 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:28:26.621 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:28:26.848 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T18:28:27.019 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:28:27.081 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:28:27.255 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T18:28:27.463 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T18:28:27.714 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T18:28:27.879 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T18:28:28.957 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:29.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:29.791 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:30.097 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:28:30.188 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:28:30.280 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:32.709 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:32.936 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:33.517 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:28:33.785 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:28:33.879 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:28:34.010 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:48.300 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:48.300 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-01-24T18:28:48.300 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-01-24T18:28:48.300 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-01-24T18:28:48.301 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-01-24T18:28:48.301 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-01-24T18:28:48.301 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-01-24T18:28:48.301 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-01-24T18:28:48.301 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-01-24T18:28:48.302 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-01-24T18:28:48.302 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:50.011 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:28:50.104 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:28:50.105 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:28:50.296 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:28:50.296 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:50.817 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:50.818 INFO:teuthology.orchestra.run.smithi186.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T18:28:50.818 INFO:teuthology.orchestra.run.smithi186.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T18:28:50.818 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:51.106 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T18:28:51.106 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:51.107 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:28:51.107 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T18:28:51.107 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:51.107 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:51.107 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T18:28:51.108 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T18:28:51.108 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T18:28:51.108 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:51.108 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T18:28:51.108 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:28:51.109 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:28:51.109 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T18:28:51.109 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T18:28:51.109 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T18:28:51.109 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T18:28:51.110 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-24T18:28:51.110 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-24T18:28:51.110 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2023-01-24T18:28:51.110 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2023-01-24T18:28:51.110 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:51.111 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T18:28:51.111 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T18:28:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:51.770 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2023-01-24T18:28:51.770 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2023-01-24T18:28:51.770 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /dev 2023-01-24T18:28:51.770 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /run 2023-01-24T18:28:51.771 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2023-01-24T18:28:51.771 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2023-01-24T18:28:51.771 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2023-01-24T18:28:51.771 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2023-01-24T18:28:51.771 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /dev 2023-01-24T18:28:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:52.493 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:52.493 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:48 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:28:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:28:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:28:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:28:53.776 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:28:53.776 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:54.276 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:54.277 INFO:teuthology.orchestra.run.smithi169.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T18:28:54.277 INFO:teuthology.orchestra.run.smithi169.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T18:28:54.277 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:54.381 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:28:54.495 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:54.496 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-01-24T18:28:54.496 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.496 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.496 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:28:54.497 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.497 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.497 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.497 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.497 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T18:28:54.498 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.498 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T18:28:54.498 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T18:28:54.498 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.498 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T18:28:54.499 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.499 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T18:28:54.499 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T18:28:54.499 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.500 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.500 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.500 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T18:28:54.500 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.500 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.501 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:54.501 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T18:28:54.501 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:54.501 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:28:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T18:28:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:28:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:28:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T18:28:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:28:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:28:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T18:28:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T18:28:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T18:28:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:28:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T18:28:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:28:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:28:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T18:28:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T18:28:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T18:28:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T18:28:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-24T18:28:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-24T18:28:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2023-01-24T18:28:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2023-01-24T18:28:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:28:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T18:28:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T18:28:54.649 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-test 2023-01-24T18:28:55.009 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:28:55.759 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:28:55.969 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:28:55.970 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:28:57.534 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:28:57.535 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================== 2023-01-24T18:28:57.536 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:28:57.536 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================== 2023-01-24T18:28:57.536 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:28:57.536 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T18:28:57.536 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:28:57.537 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T18:28:57.537 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T18:28:57.537 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T18:28:57.537 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T18:28:57.537 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T18:28:57.538 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:57.538 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:28:57.538 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================== 2023-01-24T18:28:57.538 INFO:teuthology.orchestra.run.smithi186.stdout:Install 6 Packages 2023-01-24T18:28:57.538 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:28:57.539 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 53 M 2023-01-24T18:28:57.539 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 233 M 2023-01-24T18:28:57.539 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:28:57.541 INFO:teuthology.orchestra.run.smithi186.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 20 MB/s | 70 kB 00:00 2023-01-24T18:28:57.572 INFO:teuthology.orchestra.run.smithi186.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.1 MB/s | 161 kB 00:00 2023-01-24T18:28:57.588 INFO:teuthology.orchestra.run.smithi186.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-01-24T18:28:57.604 INFO:teuthology.orchestra.run.smithi186.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 18 MB/s | 302 kB 00:00 2023-01-24T18:28:57.708 INFO:teuthology.orchestra.run.smithi186.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 1.0 MB/s | 174 kB 00:00 2023-01-24T18:28:57.886 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:28:57.996 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:57.996 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2023-01-24T18:28:57.997 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.997 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.997 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:28:57.997 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.997 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.998 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.998 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.998 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T18:28:57.998 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.998 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T18:28:57.999 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T18:28:57.999 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.999 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T18:28:57.999 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:57.999 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T18:28:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.001 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.001 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:28:58.001 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T18:28:58.001 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:28:58.001 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:28:58.138 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-test 2023-01-24T18:28:58.497 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:28:59.204 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:28:59.956 INFO:teuthology.orchestra.run.smithi186.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 22 MB/s | 52 MB 00:02 2023-01-24T18:28:59.959 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:28:59.960 INFO:teuthology.orchestra.run.smithi186.stdout:Total 22 MB/s | 53 MB 00:02 2023-01-24T18:28:59.963 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:28:59.986 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:28:59.987 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:29:00.324 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:29:00.328 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:29:00.862 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:29:01.051 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:01.053 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2023-01-24T18:29:01.053 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:29:01.053 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2023-01-24T18:29:01.054 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:29:01.054 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T18:29:01.054 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:29:01.054 INFO:teuthology.orchestra.run.smithi169.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T18:29:01.054 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T18:29:01.055 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T18:29:01.055 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T18:29:01.055 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T18:29:01.056 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:01.056 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:29:01.056 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2023-01-24T18:29:01.056 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2023-01-24T18:29:01.056 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:01.057 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 53 M 2023-01-24T18:29:01.057 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 233 M 2023-01-24T18:29:01.057 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:29:01.058 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-01-24T18:29:01.090 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.0 MB/s | 161 kB 00:00 2023-01-24T18:29:01.090 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T18:29:01.108 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 10 MB/s | 187 kB 00:00 2023-01-24T18:29:01.124 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-01-24T18:29:01.226 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 1.0 MB/s | 174 kB 00:00 2023-01-24T18:29:01.292 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:29:01.369 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:29:01.560 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T18:29:01.724 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T18:29:01.900 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:29:03.639 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 20 MB/s | 52 MB 00:02 2023-01-24T18:29:03.643 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:03.644 INFO:teuthology.orchestra.run.smithi169.stdout:Total 21 MB/s | 53 MB 00:02 2023-01-24T18:29:03.647 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:29:03.671 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:29:03.671 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:29:04.031 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:29:04.036 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:29:04.553 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:29:04.783 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T18:29:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:29:05.044 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:29:05.216 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T18:29:05.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T18:29:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:29:06.908 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:29:07.173 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:29:07.470 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:29:07.470 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T18:29:07.470 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T18:29:07.471 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T18:29:07.471 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T18:29:07.471 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T18:29:08.467 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T18:29:08.468 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:04 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:10.274 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:29:10.371 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:10.371 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:29:10.372 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:10.372 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.5-12.el8.x86_64 2023-01-24T18:29:10.372 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:10.373 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T18:29:10.373 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T18:29:10.374 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T18:29:10.374 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:10.374 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:10.485 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph 2023-01-24T18:29:10.581 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:29:10.834 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:10.861 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:29:11.143 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:29:11.144 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T18:29:11.144 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T18:29:11.144 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T18:29:11.144 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T18:29:11.145 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T18:29:11.571 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:12.176 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T18:29:12.176 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:08 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:13.368 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:13.371 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================== 2023-01-24T18:29:13.371 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:29:13.371 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================== 2023-01-24T18:29:13.371 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:29:13.371 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T18:29:13.372 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:29:13.372 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T18:29:13.373 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T18:29:13.373 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T18:29:13.373 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T18:29:13.373 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:29:13.373 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T18:29:13.374 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T18:29:13.374 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T18:29:13.374 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T18:29:13.375 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T18:29:13.375 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T18:29:13.375 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T18:29:13.375 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T18:29:13.375 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T18:29:13.376 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T18:29:13.376 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T18:29:13.376 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T18:29:13.376 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T18:29:13.376 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T18:29:13.377 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T18:29:13.378 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T18:29:13.378 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T18:29:13.378 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T18:29:13.378 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T18:29:13.379 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T18:29:13.379 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T18:29:13.379 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T18:29:13.379 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:13.379 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:29:13.380 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================== 2023-01-24T18:29:13.380 INFO:teuthology.orchestra.run.smithi186.stdout:Install 28 Packages 2023-01-24T18:29:13.380 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:13.381 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 27 M 2023-01-24T18:29:13.381 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 105 M 2023-01-24T18:29:13.381 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:29:13.473 INFO:teuthology.orchestra.run.smithi186.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 66 kB/s | 6.4 kB 00:00 2023-01-24T18:29:13.680 INFO:teuthology.orchestra.run.smithi186.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 5.1 MB/s | 1.6 MB 00:00 2023-01-24T18:29:13.794 INFO:teuthology.orchestra.run.smithi186.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 5.5 MB/s | 2.3 MB 00:00 2023-01-24T18:29:13.861 INFO:teuthology.orchestra.run.smithi186.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 3.4 MB/s | 233 kB 00:00 2023-01-24T18:29:13.913 INFO:teuthology.orchestra.run.smithi186.stdout:(5/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 8.9 MB/s | 3.9 MB 00:00 2023-01-24T18:29:13.975 INFO:teuthology.orchestra.run.smithi186.stdout:(6/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 385 kB/s | 44 kB 00:00 2023-01-24T18:29:14.039 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:29:14.050 INFO:teuthology.orchestra.run.smithi186.stdout:(7/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.3 MB/s | 185 kB 00:00 2023-01-24T18:29:14.059 INFO:teuthology.orchestra.run.smithi186.stdout:(8/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.0 MB/s | 173 kB 00:00 2023-01-24T18:29:14.088 INFO:teuthology.orchestra.run.smithi186.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 387 kB/s | 11 kB 00:00 2023-01-24T18:29:14.107 INFO:teuthology.orchestra.run.smithi186.stdout:(10/28): python3-cherrypy-18.4.0-1.el8.noarch.r 6.6 MB/s | 384 kB 00:00 2023-01-24T18:29:14.117 INFO:teuthology.orchestra.run.smithi186.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 613 kB/s | 18 kB 00:00 2023-01-24T18:29:14.135 INFO:teuthology.orchestra.run.smithi186.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.7 MB/s | 49 kB 00:00 2023-01-24T18:29:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:14.137 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:29:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: jq-1.5-12.el8.x86_64 2023-01-24T18:29:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T18:29:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T18:29:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T18:29:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:14.139 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:14.146 INFO:teuthology.orchestra.run.smithi186.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2023-01-24T18:29:14.175 INFO:teuthology.orchestra.run.smithi186.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 551 kB/s | 16 kB 00:00 2023-01-24T18:29:14.191 INFO:teuthology.orchestra.run.smithi186.stdout:(15/28): python3-pecan-1.3.2-9.el8.noarch.rpm 5.0 MB/s | 283 kB 00:00 2023-01-24T18:29:14.203 INFO:teuthology.orchestra.run.smithi186.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 667 kB/s | 19 kB 00:00 2023-01-24T18:29:14.219 INFO:teuthology.orchestra.run.smithi186.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 866 kB/s | 24 kB 00:00 2023-01-24T18:29:14.232 INFO:teuthology.orchestra.run.smithi186.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T18:29:14.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph 2023-01-24T18:29:14.246 INFO:teuthology.orchestra.run.smithi186.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 1.0 MB/s | 27 kB 00:00 2023-01-24T18:29:14.277 INFO:teuthology.orchestra.run.smithi186.stdout:(20/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 8.2 MB/s | 251 kB 00:00 2023-01-24T18:29:14.289 INFO:teuthology.orchestra.run.smithi186.stdout:(21/28): python3-waitress-1.2.1-2.el8.1.noarch. 4.1 MB/s | 241 kB 00:00 2023-01-24T18:29:14.305 INFO:teuthology.orchestra.run.smithi186.stdout:(22/28): python3-webtest-2.0.33-1.el8.noarch.rp 3.0 MB/s | 85 kB 00:00 2023-01-24T18:29:14.319 INFO:teuthology.orchestra.run.smithi186.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 756 kB/s | 23 kB 00:00 2023-01-24T18:29:14.342 INFO:teuthology.orchestra.run.smithi186.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 12 MB/s | 457 kB 00:00 2023-01-24T18:29:14.355 INFO:teuthology.orchestra.run.smithi186.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-01-24T18:29:14.363 INFO:teuthology.orchestra.run.smithi186.stdout:(26/28): python3-mako-1.0.6-13.el8.noarch.rpm 7.3 MB/s | 157 kB 00:00 2023-01-24T18:29:14.370 INFO:teuthology.orchestra.run.smithi186.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-01-24T18:29:14.515 INFO:teuthology.orchestra.run.smithi186.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 20 MB/s | 17 MB 00:00 2023-01-24T18:29:14.518 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:14.518 INFO:teuthology.orchestra.run.smithi186.stdout:Total 24 MB/s | 27 MB 00:01 2023-01-24T18:29:14.522 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:29:14.558 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:29:14.558 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:29:14.591 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:14.907 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:29:14.911 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:29:15.319 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:11 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:15.989 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:29:16.174 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T18:29:16.422 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T18:29:16.569 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T18:29:16.847 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T18:29:17.076 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:17.079 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================== 2023-01-24T18:29:17.079 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:29:17.079 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================== 2023-01-24T18:29:17.079 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:29:17.080 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T18:29:17.080 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:29:17.080 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T18:29:17.080 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T18:29:17.081 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T18:29:17.081 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T18:29:17.081 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:29:17.081 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T18:29:17.081 INFO:teuthology.orchestra.run.smithi169.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T18:29:17.082 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T18:29:17.082 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T18:29:17.082 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T18:29:17.082 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T18:29:17.083 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T18:29:17.083 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T18:29:17.083 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T18:29:17.084 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T18:29:17.084 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T18:29:17.084 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T18:29:17.085 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T18:29:17.086 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T18:29:17.086 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T18:29:17.086 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T18:29:17.086 INFO:teuthology.orchestra.run.smithi169.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T18:29:17.086 INFO:teuthology.orchestra.run.smithi169.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T18:29:17.087 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T18:29:17.087 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T18:29:17.087 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T18:29:17.087 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T18:29:17.088 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:17.088 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:29:17.088 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================== 2023-01-24T18:29:17.088 INFO:teuthology.orchestra.run.smithi169.stdout:Install 28 Packages 2023-01-24T18:29:17.088 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:17.089 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 27 M 2023-01-24T18:29:17.089 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 105 M 2023-01-24T18:29:17.090 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:29:17.191 INFO:teuthology.orchestra.run.smithi169.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 60 kB/s | 6.4 kB 00:00 2023-01-24T18:29:17.221 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T18:29:17.435 INFO:teuthology.orchestra.run.smithi169.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 4.4 MB/s | 1.6 MB 00:00 2023-01-24T18:29:17.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T18:29:17.500 INFO:teuthology.orchestra.run.smithi169.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 5.5 MB/s | 2.3 MB 00:00 2023-01-24T18:29:17.525 INFO:teuthology.orchestra.run.smithi169.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 9.0 MB/s | 233 kB 00:00 2023-01-24T18:29:17.634 INFO:teuthology.orchestra.run.smithi169.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 401 kB/s | 44 kB 00:00 2023-01-24T18:29:17.657 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T18:29:17.660 INFO:teuthology.orchestra.run.smithi169.stdout:(6/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 8.4 MB/s | 3.9 MB 00:00 2023-01-24T18:29:17.716 INFO:teuthology.orchestra.run.smithi169.stdout:(7/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.2 MB/s | 185 kB 00:00 2023-01-24T18:29:17.776 INFO:teuthology.orchestra.run.smithi169.stdout:(8/28): python3-cherrypy-18.4.0-1.el8.noarch.rp 6.3 MB/s | 384 kB 00:00 2023-01-24T18:29:17.803 INFO:teuthology.orchestra.run.smithi169.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 402 kB/s | 11 kB 00:00 2023-01-24T18:29:17.805 INFO:teuthology.orchestra.run.smithi169.stdout:(10/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.2 MB/s | 173 kB 00:00 2023-01-24T18:29:17.831 INFO:teuthology.orchestra.run.smithi169.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 638 kB/s | 18 kB 00:00 2023-01-24T18:29:17.833 INFO:teuthology.orchestra.run.smithi169.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.7 MB/s | 49 kB 00:00 2023-01-24T18:29:17.859 INFO:teuthology.orchestra.run.smithi169.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 2.1 MB/s | 59 kB 00:00 2023-01-24T18:29:17.864 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T18:29:17.887 INFO:teuthology.orchestra.run.smithi169.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 563 kB/s | 16 kB 00:00 2023-01-24T18:29:17.891 INFO:teuthology.orchestra.run.smithi169.stdout:(15/28): python3-pecan-1.3.2-9.el8.noarch.rpm 4.8 MB/s | 283 kB 00:00 2023-01-24T18:29:17.915 INFO:teuthology.orchestra.run.smithi169.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 680 kB/s | 19 kB 00:00 2023-01-24T18:29:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 850 kB/s | 24 kB 00:00 2023-01-24T18:29:17.942 INFO:teuthology.orchestra.run.smithi169.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T18:29:17.948 INFO:teuthology.orchestra.run.smithi169.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 979 kB/s | 27 kB 00:00 2023-01-24T18:29:17.973 INFO:teuthology.orchestra.run.smithi169.stdout:(20/28): python3-waitress-1.2.1-2.el8.1.noarch. 7.8 MB/s | 241 kB 00:00 2023-01-24T18:29:18.002 INFO:teuthology.orchestra.run.smithi169.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.9 MB/s | 85 kB 00:00 2023-01-24T18:29:18.004 INFO:teuthology.orchestra.run.smithi169.stdout:(22/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.4 MB/s | 251 kB 00:00 2023-01-24T18:29:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 823 kB/s | 23 kB 00:00 2023-01-24T18:29:18.036 INFO:teuthology.orchestra.run.smithi169.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 14 MB/s | 457 kB 00:00 2023-01-24T18:29:18.052 INFO:teuthology.orchestra.run.smithi169.stdout:(25/28): python3-mako-1.0.6-13.el8.noarch.rpm 9.8 MB/s | 157 kB 00:00 2023-01-24T18:29:18.056 INFO:teuthology.orchestra.run.smithi169.stdout:(26/28): python3-markupsafe-0.23-19.el8.x86_64. 1.4 MB/s | 39 kB 00:00 2023-01-24T18:29:18.064 INFO:teuthology.orchestra.run.smithi169.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 4.3 MB/s | 54 kB 00:00 2023-01-24T18:29:18.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T18:29:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T18:29:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 19 MB/s | 17 MB 00:00 2023-01-24T18:29:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:Total 22 MB/s | 27 MB 00:01 2023-01-24T18:29:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:29:18.375 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:29:18.375 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:29:18.426 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T18:29:18.660 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T18:29:18.764 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:29:18.768 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:29:18.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T18:29:19.036 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T18:29:19.254 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T18:29:19.413 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T18:29:19.688 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T18:29:19.816 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:29:19.913 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T18:29:20.063 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T18:29:20.135 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T18:29:20.309 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T18:29:20.474 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T18:29:20.478 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T18:29:20.749 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T18:29:20.761 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T18:29:21.016 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T18:29:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T18:29:21.346 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T18:29:21.470 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T18:29:21.523 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:29:21.725 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T18:29:21.953 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T18:29:22.102 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T18:29:22.291 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T18:29:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T18:29:22.723 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T18:29:22.893 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T18:29:23.000 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:29:23.116 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T18:29:23.325 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T18:29:23.541 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T18:29:23.779 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T18:29:23.815 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:29:24.027 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T18:29:24.212 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:29:24.273 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T18:29:24.397 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:29:24.591 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T18:29:24.612 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:29:24.764 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:29:24.782 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:29:24.887 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T18:29:24.930 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:29:25.208 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T18:29:25.330 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:29:25.330 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T18:29:25.330 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T18:29:25.331 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T18:29:25.331 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T18:29:25.331 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T18:29:25.331 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T18:29:25.331 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T18:29:25.332 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T18:29:25.332 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T18:29:25.332 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T18:29:25.332 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T18:29:25.332 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T18:29:25.333 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T18:29:25.333 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T18:29:25.333 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T18:29:25.333 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T18:29:25.333 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T18:29:25.334 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T18:29:25.334 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T18:29:25.334 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T18:29:25.334 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T18:29:25.335 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T18:29:25.335 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T18:29:25.335 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T18:29:25.335 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T18:29:25.335 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T18:29:25.336 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T18:29:25.601 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T18:29:25.793 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:29:26.757 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T18:29:26.757 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:22 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:27.279 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:29:27.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:29:27.860 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:29:28.070 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:29:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:29:28.435 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:29:28.452 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:29:28.577 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:29:28.592 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:29:28.677 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:28.677 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:29:28.677 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:28.677 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:28.678 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:28.678 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:29:28.678 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:28.678 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:28.678 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T18:29:28.679 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T18:29:28.679 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T18:29:28.679 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T18:29:28.679 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T18:29:28.679 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T18:29:28.680 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T18:29:28.680 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T18:29:28.680 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T18:29:28.680 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T18:29:28.680 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T18:29:28.681 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T18:29:28.681 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T18:29:28.681 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T18:29:28.681 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T18:29:28.681 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T18:29:28.682 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T18:29:28.682 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T18:29:28.682 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T18:29:28.682 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T18:29:28.682 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T18:29:28.683 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T18:29:28.683 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:28.683 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:28.819 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-base 2023-01-24T18:29:28.989 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:29:28.989 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T18:29:28.990 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T18:29:28.990 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T18:29:28.990 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T18:29:28.990 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T18:29:28.990 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T18:29:28.991 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T18:29:28.992 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T18:29:28.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T18:29:28.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T18:29:28.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T18:29:28.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T18:29:28.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T18:29:28.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T18:29:28.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T18:29:28.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T18:29:28.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T18:29:28.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T18:29:28.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T18:29:28.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T18:29:28.999 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T18:29:28.999 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T18:29:29.180 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:29.921 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:30.489 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T18:29:30.489 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:31.275 INFO:teuthology.orchestra.run.smithi186.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:29:31.725 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:31.726 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:29:31.726 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:31.834 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install cephadm 2023-01-24T18:29:32.189 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:32.349 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:29:32.447 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:32.448 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:29:32.448 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:32.448 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:32.449 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:32.449 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:29:32.449 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:32.449 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:32.449 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T18:29:32.450 INFO:teuthology.orchestra.run.smithi169.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T18:29:32.450 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T18:29:32.450 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T18:29:32.450 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T18:29:32.450 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T18:29:32.451 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T18:29:32.452 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T18:29:32.452 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T18:29:32.452 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T18:29:32.452 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T18:29:32.452 INFO:teuthology.orchestra.run.smithi169.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T18:29:32.453 INFO:teuthology.orchestra.run.smithi169.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T18:29:32.453 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T18:29:32.453 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T18:29:32.453 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T18:29:32.454 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T18:29:32.454 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:32.454 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:32.602 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-base 2023-01-24T18:29:32.938 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:28 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:32.967 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:34.697 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:34.698 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:34.699 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:29:34.699 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:34.699 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:29:34.699 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T18:29:34.699 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:34.700 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:29:34.700 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:34.700 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:29:34.700 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:34.700 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 78 k 2023-01-24T18:29:34.701 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 355 k 2023-01-24T18:29:34.701 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:29:34.861 INFO:teuthology.orchestra.run.smithi186.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 482 kB/s | 78 kB 00:00 2023-01-24T18:29:34.862 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:34.863 INFO:teuthology.orchestra.run.smithi186.stdout:Total 477 kB/s | 78 kB 00:00 2023-01-24T18:29:34.863 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:29:34.871 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:29:34.872 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:29:34.873 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:29:34.878 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:29:35.018 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:29:35.471 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:35.472 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:29:35.472 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:35.553 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:29:35.574 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install cephadm 2023-01-24T18:29:35.931 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:36.672 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:32 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:38.379 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:38.601 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:38.805 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:38.867 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:38.868 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:38.868 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:29:38.868 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:38.868 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:29:38.869 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T18:29:38.869 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:38.869 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:29:38.869 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:38.869 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:29:38.870 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:38.870 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 78 k 2023-01-24T18:29:38.870 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 355 k 2023-01-24T18:29:38.870 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:29:39.013 INFO:teuthology.orchestra.run.smithi169.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 541 kB/s | 78 kB 00:00 2023-01-24T18:29:39.014 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:39.014 INFO:teuthology.orchestra.run.smithi169.stdout:Total 536 kB/s | 78 kB 00:00 2023-01-24T18:29:39.015 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:29:39.022 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:29:39.022 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:29:39.024 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:29:39.028 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:29:39.108 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:29:39.721 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:39.721 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:35 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:41.537 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:29:41.590 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:41.632 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:41.632 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:29:41.633 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:29:41.633 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:41.633 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:41.795 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:41.802 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T18:29:42.002 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:42.171 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:42.915 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:29:42.915 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:38 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:42.933 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:38 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:44.715 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:44.716 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:44.717 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2023-01-24T18:29:44.717 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:44.717 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:29:44.717 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T18:29:44.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:44.718 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:29:44.718 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:29:44.718 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:29:44.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:44.719 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 162 k 2023-01-24T18:29:44.719 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 490 k 2023-01-24T18:29:44.719 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:29:44.886 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 956 kB/s | 162 kB 00:00 2023-01-24T18:29:44.887 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:44.888 INFO:teuthology.orchestra.run.smithi186.stdout:Total 948 kB/s | 162 kB 00:00 2023-01-24T18:29:44.888 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:29:44.898 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:29:44.900 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:29:44.900 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:29:44.953 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:29:44.957 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:29:44.995 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:44.995 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:29:44.995 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:29:44.996 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:44.996 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:45.080 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:29:45.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T18:29:45.252 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:45.510 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:45.656 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:46.331 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:42 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:46.664 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:46.664 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:42 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:48.185 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:48.186 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:48.186 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2023-01-24T18:29:48.186 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:48.186 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:29:48.187 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T18:29:48.187 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:48.187 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:29:48.187 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:29:48.188 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:29:48.188 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:48.188 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 162 k 2023-01-24T18:29:48.188 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 490 k 2023-01-24T18:29:48.189 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:29:48.366 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 902 kB/s | 162 kB 00:00 2023-01-24T18:29:48.366 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:48.367 INFO:teuthology.orchestra.run.smithi169.stdout:Total 895 kB/s | 162 kB 00:00 2023-01-24T18:29:48.367 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:29:48.380 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:29:48.380 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:29:48.432 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:29:48.436 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:29:48.525 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:29:48.560 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:29:48.620 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:48.620 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:29:48.621 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:48.621 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:48.621 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:48.757 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:48.772 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr 2023-01-24T18:29:49.131 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:49.186 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:49.847 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:45 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:50.075 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:29:50.076 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:46 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:51.173 INFO:teuthology.orchestra.run.smithi186.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:29:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:29:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:29:51.754 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T18:29:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:52.171 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:29:52.266 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:52.267 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:29:52.267 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:29:52.267 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:52.267 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:52.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr 2023-01-24T18:29:52.770 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:52.918 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:53.598 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:49 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:54.729 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:29:54.730 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================================== 2023-01-24T18:29:54.730 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:29:54.730 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================================== 2023-01-24T18:29:54.731 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:29:54.731 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T18:29:54.731 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:29:54.731 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T18:29:54.731 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T18:29:54.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:29:54.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T18:29:54.732 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T18:29:54.732 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:54.733 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:29:54.733 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================================== 2023-01-24T18:29:54.733 INFO:teuthology.orchestra.run.smithi186.stdout:Install 6 Packages 2023-01-24T18:29:54.733 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:29:54.734 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 2.0 M 2023-01-24T18:29:54.734 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 7.8 M 2023-01-24T18:29:54.734 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:29:54.870 INFO:teuthology.orchestra.run.smithi186.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 167 kB/s | 23 kB 00:00 2023-01-24T18:29:54.882 INFO:teuthology.orchestra.run.smithi186.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 97 kB/s | 14 kB 00:00 2023-01-24T18:29:54.967 INFO:teuthology.orchestra.run.smithi186.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 346 kB/s | 34 kB 00:00 2023-01-24T18:29:55.005 INFO:teuthology.orchestra.run.smithi186.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2023-01-24T18:29:55.049 INFO:teuthology.orchestra.run.smithi186.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-01-24T18:29:55.109 INFO:teuthology.orchestra.run.smithi186.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 4.4 MB/s | 1.6 MB 00:00 2023-01-24T18:29:55.111 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:55.111 INFO:teuthology.orchestra.run.smithi186.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2023-01-24T18:29:55.113 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:29:55.121 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:29:55.121 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:29:55.194 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:29:55.199 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:29:55.341 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:29:55.491 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:29:55.718 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T18:29:55.795 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:55.796 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:29:55.796 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:29:55.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T18:29:55.918 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T18:29:56.103 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T18:29:56.224 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T18:29:56.239 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:29:56.562 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T18:29:56.809 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:29:56.995 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:57.074 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:29:57.074 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T18:29:57.074 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T18:29:57.075 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T18:29:57.075 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T18:29:57.075 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T18:29:58.012 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T18:29:58.013 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:54 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:29:58.762 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:29:58.763 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================================== 2023-01-24T18:29:58.763 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:29:58.763 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================================== 2023-01-24T18:29:58.763 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:29:58.764 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T18:29:58.764 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:29:58.764 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T18:29:58.764 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T18:29:58.765 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:29:58.765 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T18:29:58.765 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T18:29:58.765 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:58.765 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:29:58.766 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================================== 2023-01-24T18:29:58.766 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2023-01-24T18:29:58.766 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:29:58.767 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.0 M 2023-01-24T18:29:58.767 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 7.8 M 2023-01-24T18:29:58.767 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:29:58.905 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 164 kB/s | 23 kB 00:00 2023-01-24T18:29:58.918 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 94 kB/s | 14 kB 00:00 2023-01-24T18:29:58.987 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 412 kB/s | 34 kB 00:00 2023-01-24T18:29:59.018 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-01-24T18:29:59.060 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 5.6 MB/s | 1.6 MB 00:00 2023-01-24T18:29:59.088 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-01-24T18:29:59.091 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:29:59.091 INFO:teuthology.orchestra.run.smithi169.stdout:Total 6.0 MB/s | 2.0 MB 00:00 2023-01-24T18:29:59.092 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:29:59.100 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:29:59.100 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:29:59.184 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:29:59.188 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:29:59.473 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:29:59.685 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T18:29:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T18:29:59.977 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:00.072 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:00.073 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:00.073 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:00.073 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:00.073 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:00.073 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T18:30:00.074 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T18:30:00.074 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T18:30:00.076 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:00.076 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T18:30:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T18:30:00.209 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T18:30:00.571 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:01.322 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:57 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:01.469 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T18:30:01.703 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:30:01.992 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:30:01.992 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T18:30:01.992 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T18:30:01.992 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T18:30:01.993 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T18:30:01.993 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T18:30:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T18:30:03.033 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:59 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:03.093 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:03.094 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================================== 2023-01-24T18:30:03.094 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:30:03.095 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================================== 2023-01-24T18:30:03.095 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:03.095 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T18:30:03.095 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:30:03.095 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T18:30:03.096 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T18:30:03.096 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T18:30:03.096 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T18:30:03.096 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T18:30:03.096 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T18:30:03.097 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T18:30:03.097 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:03.097 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:03.097 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================================================== 2023-01-24T18:30:03.097 INFO:teuthology.orchestra.run.smithi186.stdout:Install 8 Packages 2023-01-24T18:30:03.098 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:03.098 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 36 M 2023-01-24T18:30:03.098 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 199 M 2023-01-24T18:30:03.098 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:03.146 INFO:teuthology.orchestra.run.smithi186.stdout:(1/8): libquadmath-8.4.1-1.el8.x86_64.rpm 3.3 MB/s | 169 kB 00:00 2023-01-24T18:30:03.152 INFO:teuthology.orchestra.run.smithi186.stdout:(2/8): libgfortran-8.4.1-1.el8.x86_64.rpm 11 MB/s | 639 kB 00:00 2023-01-24T18:30:03.169 INFO:teuthology.orchestra.run.smithi186.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 13 MB/s | 225 kB 00:00 2023-01-24T18:30:03.231 INFO:teuthology.orchestra.run.smithi186.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 61 MB/s | 3.7 MB 00:00 2023-01-24T18:30:03.308 INFO:teuthology.orchestra.run.smithi186.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 61 MB/s | 4.7 MB 00:00 2023-01-24T18:30:03.443 INFO:teuthology.orchestra.run.smithi186.stdout:(6/8): python3-scipy-1.0.0-20.module+el8.1.0+33 49 MB/s | 15 MB 00:00 2023-01-24T18:30:03.485 INFO:teuthology.orchestra.run.smithi186.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 26 MB/s | 4.6 MB 00:00 2023-01-24T18:30:03.886 INFO:teuthology.orchestra.run.smithi186.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 9.4 MB/s | 7.4 MB 00:00 2023-01-24T18:30:03.888 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:03.888 INFO:teuthology.orchestra.run.smithi186.stdout:Total 45 MB/s | 36 MB 00:00 2023-01-24T18:30:03.897 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:03.916 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:03.916 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:04.319 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:04.323 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:04.914 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:04.940 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:05.012 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:05.012 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:05.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:05.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:05.013 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:05.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T18:30:05.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T18:30:05.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T18:30:05.013 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:05.014 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:05.133 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:30:05.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T18:30:05.274 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:30:05.447 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:05.494 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:05.996 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:06.182 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:30:06.378 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:02 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:06.719 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:30:06.932 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:30:07.655 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:30:07.968 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:30:08.199 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:08.221 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:08.222 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================================== 2023-01-24T18:30:08.223 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:30:08.223 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================================== 2023-01-24T18:30:08.223 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:08.223 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T18:30:08.223 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:30:08.224 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T18:30:08.224 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T18:30:08.224 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T18:30:08.224 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T18:30:08.224 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T18:30:08.225 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T18:30:08.225 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T18:30:08.225 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:08.225 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:08.225 INFO:teuthology.orchestra.run.smithi169.stdout:====================================================================================================================== 2023-01-24T18:30:08.226 INFO:teuthology.orchestra.run.smithi169.stdout:Install 8 Packages 2023-01-24T18:30:08.226 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:08.226 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 36 M 2023-01-24T18:30:08.227 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 199 M 2023-01-24T18:30:08.227 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:08.257 INFO:teuthology.orchestra.run.smithi169.stdout:(1/8): libquadmath-8.4.1-1.el8.x86_64.rpm 5.1 MB/s | 169 kB 00:00 2023-01-24T18:30:08.272 INFO:teuthology.orchestra.run.smithi169.stdout:(2/8): libgfortran-8.4.1-1.el8.x86_64.rpm 13 MB/s | 639 kB 00:00 2023-01-24T18:30:08.289 INFO:teuthology.orchestra.run.smithi169.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 13 MB/s | 225 kB 00:00 2023-01-24T18:30:08.358 INFO:teuthology.orchestra.run.smithi169.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 54 MB/s | 3.7 MB 00:00 2023-01-24T18:30:08.442 INFO:teuthology.orchestra.run.smithi169.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 57 MB/s | 4.7 MB 00:00 2023-01-24T18:30:08.577 INFO:teuthology.orchestra.run.smithi169.stdout:(6/8): python3-scipy-1.0.0-20.module+el8.1.0+33 46 MB/s | 15 MB 00:00 2023-01-24T18:30:08.623 INFO:teuthology.orchestra.run.smithi169.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 25 MB/s | 4.6 MB 00:00 2023-01-24T18:30:09.168 INFO:teuthology.orchestra.run.smithi169.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 7.8 MB/s | 7.4 MB 00:00 2023-01-24T18:30:09.169 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:09.170 INFO:teuthology.orchestra.run.smithi169.stdout:Total 38 MB/s | 36 MB 00:00 2023-01-24T18:30:09.179 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:09.198 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:09.198 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:09.625 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:09.629 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:10.240 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:10.256 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:10.411 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:30:10.552 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:30:10.686 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:11.228 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:11.402 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:30:11.520 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T18:30:11.690 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:30:11.966 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:30:11.988 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:30:11.988 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T18:30:11.988 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:11.988 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 3/8 2023-01-24T18:30:11.989 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2023-01-24T18:30:11.989 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:30:11.989 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:11.989 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 7/8 2023-01-24T18:30:12.178 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:30:12.915 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:30:12.930 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 8/8 2023-01-24T18:30:12.930 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:08 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:13.254 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:30:13.477 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:14.770 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:14.867 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:14.867 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:14.867 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:14.867 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T18:30:14.868 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T18:30:14.868 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T18:30:14.868 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T18:30:14.868 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T18:30:14.868 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T18:30:14.869 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T18:30:14.869 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:14.869 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:15.025 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-rook 2023-01-24T18:30:15.388 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:15.551 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:16.133 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:12 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T18:30:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:30:17.281 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:30:17.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T18:30:17.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:30:17.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 3/8 2023-01-24T18:30:17.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2023-01-24T18:30:17.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:30:17.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:30:17.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 7/8 2023-01-24T18:30:17.927 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:17.929 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================================= 2023-01-24T18:30:17.929 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:30:17.929 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================================= 2023-01-24T18:30:17.929 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:17.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T18:30:17.930 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:30:17.930 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T18:30:17.930 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T18:30:17.930 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T18:30:17.931 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T18:30:17.931 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T18:30:17.931 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T18:30:17.931 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T18:30:17.931 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T18:30:17.932 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T18:30:17.932 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:30:17.932 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T18:30:17.932 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T18:30:17.933 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:17.933 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:17.933 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================================= 2023-01-24T18:30:17.933 INFO:teuthology.orchestra.run.smithi186.stdout:Install 13 Packages 2023-01-24T18:30:17.933 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:17.934 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 1.9 M 2023-01-24T18:30:17.934 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 25 M 2023-01-24T18:30:17.934 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:18.091 INFO:teuthology.orchestra.run.smithi186.stdout:(1/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 295 kB/s | 47 kB 00:00 2023-01-24T18:30:18.198 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 8/8 2023-01-24T18:30:18.199 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:14 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:20.154 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:20.254 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:20.255 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:20.255 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:20.255 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T18:30:20.255 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T18:30:20.256 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T18:30:20.256 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T18:30:20.256 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T18:30:20.256 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T18:30:20.256 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T18:30:20.257 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:20.257 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:20.390 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-rook 2023-01-24T18:30:20.403 INFO:teuthology.orchestra.run.smithi186.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 6.6 kB/s | 16 kB 00:02 2023-01-24T18:30:20.405 INFO:teuthology.orchestra.run.smithi186.stdout:(3/13): python3-cachetools-3.1.1-4.el8.noarch.r 13 kB/s | 33 kB 00:02 2023-01-24T18:30:20.430 INFO:teuthology.orchestra.run.smithi186.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 35 kB/s | 81 kB 00:02 2023-01-24T18:30:20.457 INFO:teuthology.orchestra.run.smithi186.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 2.2 MB/s | 61 kB 00:00 2023-01-24T18:30:20.462 INFO:teuthology.orchestra.run.smithi186.stdout:(6/13): python3-rsa-4.8-1.el8.noarch.rpm 1.1 MB/s | 61 kB 00:00 2023-01-24T18:30:20.506 INFO:teuthology.orchestra.run.smithi186.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.1 MB/s | 155 kB 00:00 2023-01-24T18:30:20.510 INFO:teuthology.orchestra.run.smithi186.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 893 kB/s | 43 kB 00:00 2023-01-24T18:30:20.522 INFO:teuthology.orchestra.run.smithi186.stdout:(9/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 1.6 MB/s | 27 kB 00:00 2023-01-24T18:30:20.531 INFO:teuthology.orchestra.run.smithi186.stdout:(10/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 6.0 MB/s | 126 kB 00:00 2023-01-24T18:30:20.538 INFO:teuthology.orchestra.run.smithi186.stdout:(11/13): python3-pyasn1-modules-0.3.7-6.el8.noa 7.1 MB/s | 110 kB 00:00 2023-01-24T18:30:20.544 INFO:teuthology.orchestra.run.smithi186.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 1.5 MB/s | 20 kB 00:00 2023-01-24T18:30:20.749 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:20.773 INFO:teuthology.orchestra.run.smithi186.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.1 MB/s | 1.2 MB 00:00 2023-01-24T18:30:20.774 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:20.774 INFO:teuthology.orchestra.run.smithi186.stdout:Total 694 kB/s | 1.9 MB 00:02 2023-01-24T18:30:20.781 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:20.797 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:20.797 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:20.886 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:20.890 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:21.418 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:21.502 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:17 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:21.668 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T18:30:21.936 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T18:30:22.154 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T18:30:22.346 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-24T18:30:22.597 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-24T18:30:22.837 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T18:30:23.128 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T18:30:23.297 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:23.299 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================================= 2023-01-24T18:30:23.299 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:30:23.299 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================================= 2023-01-24T18:30:23.299 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T18:30:23.300 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:30:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T18:30:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T18:30:23.300 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T18:30:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T18:30:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T18:30:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T18:30:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T18:30:23.301 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T18:30:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T18:30:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:30:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T18:30:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T18:30:23.302 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:23.303 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:23.303 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================================= 2023-01-24T18:30:23.303 INFO:teuthology.orchestra.run.smithi169.stdout:Install 13 Packages 2023-01-24T18:30:23.303 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:23.304 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 1.9 M 2023-01-24T18:30:23.304 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 25 M 2023-01-24T18:30:23.304 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:23.383 INFO:teuthology.orchestra.run.smithi169.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 202 kB/s | 16 kB 00:00 2023-01-24T18:30:23.384 INFO:teuthology.orchestra.run.smithi169.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 399 kB/s | 33 kB 00:00 2023-01-24T18:30:23.406 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T18:30:23.436 INFO:teuthology.orchestra.run.smithi169.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.5 MB/s | 81 kB 00:00 2023-01-24T18:30:23.444 INFO:teuthology.orchestra.run.smithi169.stdout:(4/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 329 kB/s | 47 kB 00:00 2023-01-24T18:30:23.463 INFO:teuthology.orchestra.run.smithi169.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 2.2 MB/s | 61 kB 00:00 2023-01-24T18:30:23.497 INFO:teuthology.orchestra.run.smithi169.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 4.5 MB/s | 155 kB 00:00 2023-01-24T18:30:23.513 INFO:teuthology.orchestra.run.smithi169.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 2.7 MB/s | 43 kB 00:00 2023-01-24T18:30:23.526 INFO:teuthology.orchestra.run.smithi169.stdout:(8/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 2.0 MB/s | 27 kB 00:00 2023-01-24T18:30:23.540 INFO:teuthology.orchestra.run.smithi169.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.0 MB/s | 126 kB 00:00 2023-01-24T18:30:23.554 INFO:teuthology.orchestra.run.smithi169.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 7.8 MB/s | 110 kB 00:00 2023-01-24T18:30:23.562 INFO:teuthology.orchestra.run.smithi169.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 523 kB/s | 61 kB 00:00 2023-01-24T18:30:23.566 INFO:teuthology.orchestra.run.smithi169.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 1.6 MB/s | 20 kB 00:00 2023-01-24T18:30:23.581 INFO:teuthology.orchestra.run.smithi169.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 5.9 MB/s | 1.2 MB 00:00 2023-01-24T18:30:23.581 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:23.582 INFO:teuthology.orchestra.run.smithi169.stdout:Total 6.9 MB/s | 1.9 MB 00:00 2023-01-24T18:30:23.587 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:23.601 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:23.601 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T18:30:23.690 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:23.694 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:23.905 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T18:30:24.236 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T18:30:24.773 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T18:30:25.016 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T18:30:25.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T18:30:25.233 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-24T18:30:25.483 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T18:30:25.509 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-24T18:30:25.621 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:30:25.808 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T18:30:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:30:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T18:30:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T18:30:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T18:30:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T18:30:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T18:30:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T18:30:25.891 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T18:30:25.891 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T18:30:25.891 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T18:30:25.891 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2023-01-24T18:30:25.892 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2023-01-24T18:30:25.892 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/13 2023-01-24T18:30:26.098 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T18:30:26.359 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T18:30:26.562 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T18:30:26.836 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T18:30:26.905 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-24T18:30:26.906 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:22 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:28.052 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T18:30:28.352 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T18:30:28.508 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:30:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:30:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T18:30:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T18:30:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T18:30:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T18:30:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T18:30:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T18:30:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T18:30:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T18:30:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T18:30:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2023-01-24T18:30:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2023-01-24T18:30:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/13 2023-01-24T18:30:28.969 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:29.065 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:29.066 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:29.066 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:29.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T18:30:29.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T18:30:29.066 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T18:30:29.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T18:30:29.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T18:30:29.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T18:30:29.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T18:30:29.067 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T18:30:29.068 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T18:30:29.068 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T18:30:29.068 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T18:30:29.068 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T18:30:29.068 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:29.069 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:29.204 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T18:30:29.571 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:29.786 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-24T18:30:29.787 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:25 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:30.348 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:26 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:31.697 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:31.795 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:31.796 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:31.796 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:31.796 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T18:30:31.796 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T18:30:31.797 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T18:30:31.797 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T18:30:31.797 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T18:30:31.797 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T18:30:31.797 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T18:30:31.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T18:30:31.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T18:30:31.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T18:30:31.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T18:30:31.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T18:30:31.799 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:31.799 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:31.933 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T18:30:32.100 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:32.101 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================== 2023-01-24T18:30:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:30:32.101 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================== 2023-01-24T18:30:32.101 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:32.101 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T18:30:32.102 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:30:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T18:30:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T18:30:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T18:30:32.102 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T18:30:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:32.103 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:32.103 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================== 2023-01-24T18:30:32.103 INFO:teuthology.orchestra.run.smithi186.stdout:Install 5 Packages 2023-01-24T18:30:32.103 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:32.104 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 5.9 M 2023-01-24T18:30:32.104 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 26 M 2023-01-24T18:30:32.104 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:32.213 INFO:teuthology.orchestra.run.smithi186.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 549 kB/s | 60 kB 00:00 2023-01-24T18:30:32.294 INFO:teuthology.orchestra.run.smithi186.stdout:(2/5): python3-babel-2.5.1-5.el8.noarch.rpm 59 MB/s | 4.8 MB 00:00 2023-01-24T18:30:32.296 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:32.299 INFO:teuthology.orchestra.run.smithi186.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.4 MB/s | 476 kB 00:00 2023-01-24T18:30:32.311 INFO:teuthology.orchestra.run.smithi186.stdout:(4/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 30 MB/s | 538 kB 00:00 2023-01-24T18:30:32.344 INFO:teuthology.orchestra.run.smithi186.stdout:(5/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 476 kB/s | 115 kB 00:00 2023-01-24T18:30:32.346 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:32.346 INFO:teuthology.orchestra.run.smithi186.stdout:Total 24 MB/s | 5.9 MB 00:00 2023-01-24T18:30:32.349 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:32.359 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:32.359 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:32.460 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:32.464 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:33.035 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:29 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:33.458 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:33.800 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T18:30:34.049 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T18:30:34.428 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:30:34.716 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T18:30:34.796 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================== 2023-01-24T18:30:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:30:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================== 2023-01-24T18:30:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T18:30:34.798 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:30:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T18:30:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T18:30:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T18:30:34.799 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T18:30:34.799 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:34.799 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:34.799 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================== 2023-01-24T18:30:34.799 INFO:teuthology.orchestra.run.smithi169.stdout:Install 5 Packages 2023-01-24T18:30:34.800 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:34.800 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 5.9 M 2023-01-24T18:30:34.801 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 26 M 2023-01-24T18:30:34.801 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:34.912 INFO:teuthology.orchestra.run.smithi169.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 534 kB/s | 60 kB 00:00 2023-01-24T18:30:34.966 INFO:teuthology.orchestra.run.smithi169.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 691 kB/s | 115 kB 00:00 2023-01-24T18:30:35.001 INFO:teuthology.orchestra.run.smithi169.stdout:(3/5): python3-babel-2.5.1-5.el8.noarch.rpm 54 MB/s | 4.8 MB 00:00 2023-01-24T18:30:35.006 INFO:teuthology.orchestra.run.smithi169.stdout:(4/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 13 MB/s | 538 kB 00:00 2023-01-24T18:30:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2023-01-24T18:30:35.159 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:35.160 INFO:teuthology.orchestra.run.smithi169.stdout:Total 16 MB/s | 5.9 MB 00:00 2023-01-24T18:30:35.162 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:35.172 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:35.173 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:35.263 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:35.267 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:35.336 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:30:35.604 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:30:35.605 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T18:30:35.605 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T18:30:35.605 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:30:35.605 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T18:30:36.302 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:36.524 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T18:30:36.524 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:32 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:36.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T18:30:36.902 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T18:30:37.247 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:30:37.520 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T18:30:37.698 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:30:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:30:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T18:30:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T18:30:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:30:37.967 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T18:30:38.371 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:38.469 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:38.470 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:38.470 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:38.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T18:30:38.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T18:30:38.471 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T18:30:38.471 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T18:30:38.471 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:38.471 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:38.597 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-fuse 2023-01-24T18:30:38.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T18:30:38.893 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:34 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:38.965 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:39.699 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:40.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:40.853 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T18:30:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T18:30:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T18:30:40.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T18:30:40.854 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:40.854 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:40.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-fuse 2023-01-24T18:30:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:42.086 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:38 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:42.120 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:42.121 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:42.121 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:30:42.122 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:42.122 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:42.122 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse 2023-01-24T18:30:42.122 INFO:teuthology.orchestra.run.smithi186.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T18:30:42.122 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-01-24T18:30:42.123 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T18:30:42.123 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:42.123 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:42.123 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:42.123 INFO:teuthology.orchestra.run.smithi186.stdout:Install 2 Packages 2023-01-24T18:30:42.124 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:42.124 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 954 k 2023-01-24T18:30:42.124 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 3.0 M 2023-01-24T18:30:42.125 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:42.162 INFO:teuthology.orchestra.run.smithi186.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-01-24T18:30:42.360 INFO:teuthology.orchestra.run.smithi186.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 872 kB 00:00 2023-01-24T18:30:42.360 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:42.361 INFO:teuthology.orchestra.run.smithi186.stdout:Total 3.9 MB/s | 954 kB 00:00 2023-01-24T18:30:42.362 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:42.383 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:42.384 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:42.466 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:42.470 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:42.631 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:42.879 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T18:30:43.086 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:30:43.462 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:30:43.463 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T18:30:44.114 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:44.115 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:44.115 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:30:44.115 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:44.115 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:44.115 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse 2023-01-24T18:30:44.116 INFO:teuthology.orchestra.run.smithi169.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T18:30:44.116 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-01-24T18:30:44.116 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T18:30:44.116 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:44.117 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:44.117 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:44.117 INFO:teuthology.orchestra.run.smithi169.stdout:Install 2 Packages 2023-01-24T18:30:44.117 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:44.118 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 954 k 2023-01-24T18:30:44.118 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 3.0 M 2023-01-24T18:30:44.118 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:44.145 INFO:teuthology.orchestra.run.smithi169.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.8 MB/s | 83 kB 00:00 2023-01-24T18:30:44.352 INFO:teuthology.orchestra.run.smithi169.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 872 kB 00:00 2023-01-24T18:30:44.352 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:44.353 INFO:teuthology.orchestra.run.smithi169.stdout:Total 3.9 MB/s | 954 kB 00:00 2023-01-24T18:30:44.354 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:44.375 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:44.376 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:44.414 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T18:30:44.415 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:40 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:44.665 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:44.965 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T18:30:45.148 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:30:45.525 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:30:45.525 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T18:30:46.364 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:46.460 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:46.461 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:46.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T18:30:46.461 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:46.461 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:46.477 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T18:30:46.477 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:42 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:46.588 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-volume 2023-01-24T18:30:46.958 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:47.673 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:43 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:48.319 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:48.416 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:48.417 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:48.417 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T18:30:48.417 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:48.417 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:48.531 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-volume 2023-01-24T18:30:48.904 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:49.447 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:49.448 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:49.448 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-01-24T18:30:49.448 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:49.449 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:49.449 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T18:30:49.449 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:49.449 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:49.449 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:49.450 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:30:49.450 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:49.450 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 263 k 2023-01-24T18:30:49.450 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 1.3 M 2023-01-24T18:30:49.450 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:49.669 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:45 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:50.152 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 374 kB/s | 263 kB 00:00 2023-01-24T18:30:50.152 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:50.153 INFO:teuthology.orchestra.run.smithi186.stdout:Total 373 kB/s | 263 kB 00:00 2023-01-24T18:30:50.153 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:50.163 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:50.164 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:50.190 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:50.194 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:50.374 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:50.651 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:51.040 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:51.479 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:51.480 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:51.481 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-01-24T18:30:51.481 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:51.481 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:51.481 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T18:30:51.481 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:51.482 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:51.482 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:51.482 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:30:51.482 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:51.482 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 263 k 2023-01-24T18:30:51.483 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 1.3 M 2023-01-24T18:30:51.483 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:51.785 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 864 kB/s | 263 kB 00:00 2023-01-24T18:30:51.786 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:51.786 INFO:teuthology.orchestra.run.smithi169.stdout:Total 860 kB/s | 263 kB 00:00 2023-01-24T18:30:51.787 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:51.797 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:51.797 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:51.821 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:51.825 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:51.938 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:47 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:52.030 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:52.338 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:52.732 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:53.706 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:30:53.706 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:49 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:53.801 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:30:53.896 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:53.896 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:30:53.896 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:53.896 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:53.897 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:30:54.022 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librados-devel 2023-01-24T18:30:54.391 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:55.146 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:51 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:55.587 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:30:55.683 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:55.683 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:30:55.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:30:55.684 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:55.684 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:30:55.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados-devel 2023-01-24T18:30:56.167 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:30:56.876 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:52 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:56.988 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:30:56.989 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:56.990 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2023-01-24T18:30:56.990 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:56.990 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:30:56.990 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T18:30:56.990 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 130 k 2023-01-24T18:30:56.991 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 464 k 2023-01-24T18:30:56.992 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:30:57.157 INFO:teuthology.orchestra.run.smithi186.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 781 kB/s | 130 kB 00:00 2023-01-24T18:30:57.157 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:57.158 INFO:teuthology.orchestra.run.smithi186.stdout:Total 774 kB/s | 130 kB 00:00 2023-01-24T18:30:57.158 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:30:57.164 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:30:57.165 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:30:57.221 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:30:57.225 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:30:57.344 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:30:57.542 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:30:57.817 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:30:58.725 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:30:58.726 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:58.726 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:30:58.727 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:30:58.728 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:30:58.728 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:30:58.728 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 130 k 2023-01-24T18:30:58.728 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 464 k 2023-01-24T18:30:58.728 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:30:58.892 INFO:teuthology.orchestra.run.smithi169.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 785 kB/s | 130 kB 00:00 2023-01-24T18:30:58.893 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:30:58.893 INFO:teuthology.orchestra.run.smithi169.stdout:Total 778 kB/s | 130 kB 00:00 2023-01-24T18:30:58.894 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:30:58.900 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:30:58.900 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:30:58.955 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:30:58.959 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:30:59.010 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:30:59.010 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:55 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:30:59.117 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:30:59.269 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:30:59.537 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:00.459 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:00.460 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:56 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:00.975 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:31:01.072 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:01.073 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:31:01.073 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:01.073 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:01.073 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:01.228 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs2 2023-01-24T18:31:01.596 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:02.367 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:58 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:02.489 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:31:02.585 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:02.586 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:31:02.586 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:02.586 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:02.587 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:02.726 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs2 2023-01-24T18:31:03.095 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:03.779 INFO:teuthology.orchestra.run.smithi186.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:03.853 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:59 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:04.228 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:04.229 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:04.230 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:04.351 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs-devel 2023-01-24T18:31:04.717 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:05.204 INFO:teuthology.orchestra.run.smithi169.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:05.440 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:01 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:05.659 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:05.660 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:05.661 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:05.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs-devel 2023-01-24T18:31:06.128 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:06.860 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:02 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:07.197 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:07.198 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:07.198 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2023-01-24T18:31:07.198 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:07.199 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:31:07.199 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T18:31:07.199 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:07.199 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:31:07.200 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:07.200 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:31:07.200 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:07.200 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 25 k 2023-01-24T18:31:07.200 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 109 k 2023-01-24T18:31:07.201 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:31:07.319 INFO:teuthology.orchestra.run.smithi186.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 206 kB/s | 25 kB 00:00 2023-01-24T18:31:07.319 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:07.320 INFO:teuthology.orchestra.run.smithi186.stdout:Total 204 kB/s | 25 kB 00:00 2023-01-24T18:31:07.320 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:31:07.324 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:31:07.324 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:31:07.325 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:31:07.330 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:31:07.391 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:31:07.556 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:07.777 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:08.663 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:08.663 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:04 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:08.681 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2023-01-24T18:31:08.683 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:08.683 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:31:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T18:31:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:08.683 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:08.685 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 25 k 2023-01-24T18:31:08.685 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 109 k 2023-01-24T18:31:08.685 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:31:08.808 INFO:teuthology.orchestra.run.smithi169.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 199 kB/s | 25 kB 00:00 2023-01-24T18:31:08.808 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:08.809 INFO:teuthology.orchestra.run.smithi169.stdout:Total 197 kB/s | 25 kB 00:00 2023-01-24T18:31:08.809 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:31:08.813 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:31:08.813 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:31:08.814 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:31:08.819 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:31:08.873 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:31:09.050 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:09.298 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:10.179 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:10.179 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:06 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:10.483 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:31:10.580 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:10.581 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:31:10.581 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:10.581 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:10.581 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:10.716 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librados2 2023-01-24T18:31:11.081 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:11.992 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:07 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:12.194 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:31:12.292 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:12.292 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:31:12.292 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:12.293 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:12.293 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:12.410 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados2 2023-01-24T18:31:12.777 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:13.310 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:13.575 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:09 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:13.766 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:13.768 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:13.768 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:13.871 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librbd1 2023-01-24T18:31:14.237 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:14.944 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:15.168 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:11 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:15.396 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:15.397 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:15.397 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:15.494 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librbd1 2023-01-24T18:31:15.862 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:16.460 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:16.693 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:12 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:16.906 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:16.907 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:16.907 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:17.019 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rados 2023-01-24T18:31:17.382 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:18.085 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:18.116 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:14 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:18.543 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:18.545 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:18.545 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:18.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rados 2023-01-24T18:31:19.013 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:19.518 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:19.753 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:15 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:19.969 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:19.970 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:19.970 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:20.079 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rgw 2023-01-24T18:31:20.448 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:21.085 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:21.316 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:17 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:21.540 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:21.541 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:21.541 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:21.641 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rgw 2023-01-24T18:31:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:22.634 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:22.737 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:18 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:23.094 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:23.095 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:23.095 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:23.202 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-cephfs 2023-01-24T18:31:23.567 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:24.143 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:24.287 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:20 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:24.599 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:24.601 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:24.601 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:24.700 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-cephfs 2023-01-24T18:31:25.066 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:25.683 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:25.909 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:21 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:26.152 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:26.153 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:26.154 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:26.265 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rbd 2023-01-24T18:31:26.631 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:27.276 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:27.364 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:23 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:27.730 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:27.732 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:27.732 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:27.831 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rbd 2023-01-24T18:31:28.197 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:28.681 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:28.929 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:24 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:29.134 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:29.135 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2023-01-24T18:31:29.135 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:29.244 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-fuse 2023-01-24T18:31:29.612 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:30.283 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:31:30.369 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:26 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:30.736 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:30.737 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2023-01-24T18:31:30.738 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:30.831 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-fuse 2023-01-24T18:31:31.198 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:27 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:32.149 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:32.150 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:32.151 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-01-24T18:31:32.151 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:32.151 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:31:32.151 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T18:31:32.151 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:32.152 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:31:32.152 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:32.152 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:31:32.152 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:32.152 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 90 k 2023-01-24T18:31:32.153 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 264 k 2023-01-24T18:31:32.153 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:31:33.130 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 92 kB/s | 90 kB 00:00 2023-01-24T18:31:33.131 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:33.132 INFO:teuthology.orchestra.run.smithi186.stdout:Total 92 kB/s | 90 kB 00:00 2023-01-24T18:31:33.133 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:31:33.146 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:31:33.146 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:31:33.209 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:31:33.214 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:31:33.333 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:31:33.536 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:33.730 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2023-01-24T18:31:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:31:33.731 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T18:31:33.731 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:31:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:31:33.732 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 90 k 2023-01-24T18:31:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 264 k 2023-01-24T18:31:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:31:33.810 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:34.034 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 297 kB/s | 90 kB 00:00 2023-01-24T18:31:34.035 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:34.035 INFO:teuthology.orchestra.run.smithi169.stdout:Total 295 kB/s | 90 kB 00:00 2023-01-24T18:31:34.036 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:31:34.047 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:31:34.047 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:31:34.102 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:31:34.106 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:31:34.219 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:31:34.421 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:34.688 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:34.793 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:34.793 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:30 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:35.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:35.582 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:31 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:36.620 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:31:36.715 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:36.716 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:31:36.716 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:36.716 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:36.717 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:36.837 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-mirror 2023-01-24T18:31:37.205 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:37.495 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:31:37.595 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:37.595 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:31:37.598 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:37.598 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:37.599 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:37.704 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-mirror 2023-01-24T18:31:37.945 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:33 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:38.071 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:38.805 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:34 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:39.704 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:39.705 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:39.705 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2023-01-24T18:31:39.705 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:39.706 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:31:39.706 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:31:39.706 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:39.706 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:31:39.706 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:39.707 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:31:39.707 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:39.707 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 3.5 M 2023-01-24T18:31:39.707 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 16 M 2023-01-24T18:31:39.707 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:31:40.395 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 5.0 MB/s | 3.5 MB 00:00 2023-01-24T18:31:40.396 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:40.396 INFO:teuthology.orchestra.run.smithi186.stdout:Total 5.0 MB/s | 3.5 MB 00:00 2023-01-24T18:31:40.397 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:31:40.410 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:31:40.411 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:31:40.478 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:31:40.483 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:31:40.573 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:40.574 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:40.574 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2023-01-24T18:31:40.574 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:40.575 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:31:40.575 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:31:40.575 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:40.575 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:31:40.575 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:40.576 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:31:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:40.576 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 3.5 M 2023-01-24T18:31:40.576 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 16 M 2023-01-24T18:31:40.576 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:31:40.902 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 11 MB/s | 3.5 MB 00:00 2023-01-24T18:31:40.903 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:40.903 INFO:teuthology.orchestra.run.smithi169.stdout:Total 11 MB/s | 3.5 MB 00:00 2023-01-24T18:31:40.904 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:31:40.917 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:31:40.917 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:31:40.928 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:31:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:31:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:31:41.111 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:41.451 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:31:41.526 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:41.672 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:42.089 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:42.451 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:42.451 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:38 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:43.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:43.027 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:39 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:44.463 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:31:44.558 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:44.559 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:31:44.559 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:44.559 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:44.559 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:44.698 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-nbd 2023-01-24T18:31:44.862 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:31:44.960 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:44.960 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:31:44.960 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:44.960 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:44.961 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:45.065 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:45.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-nbd 2023-01-24T18:31:45.471 INFO:teuthology.orchestra.run.smithi169.stdout:Updating Subscription Management repositories. 2023-01-24T18:31:45.831 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:41 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:46.221 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:42 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:47.614 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-01-24T18:31:47.615 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-01-24T18:31:47.615 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:47.615 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-01-24T18:31:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T18:31:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:47.616 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-01-24T18:31:47.616 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-01-24T18:31:47.616 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2023-01-24T18:31:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:47.617 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 172 k 2023-01-24T18:31:47.617 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 548 k 2023-01-24T18:31:47.617 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-01-24T18:31:47.784 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2023-01-24T18:31:47.784 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:47.785 INFO:teuthology.orchestra.run.smithi186.stdout:Total 1.0 MB/s | 172 kB 00:00 2023-01-24T18:31:47.785 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-01-24T18:31:47.798 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-01-24T18:31:47.798 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-01-24T18:31:47.854 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-01-24T18:31:47.858 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-01-24T18:31:47.980 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-01-24T18:31:48.055 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-01-24T18:31:48.056 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:48.056 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2023-01-24T18:31:48.056 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:48.057 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-01-24T18:31:48.057 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T18:31:48.057 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:48.057 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-01-24T18:31:48.057 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-01-24T18:31:48.058 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2023-01-24T18:31:48.058 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:48.058 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 172 k 2023-01-24T18:31:48.058 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 548 k 2023-01-24T18:31:48.058 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-01-24T18:31:48.175 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:48.223 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2023-01-24T18:31:48.224 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:31:48.224 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.0 MB/s | 172 kB 00:00 2023-01-24T18:31:48.225 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-01-24T18:31:48.237 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-01-24T18:31:48.237 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-01-24T18:31:48.292 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-01-24T18:31:48.297 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-01-24T18:31:48.426 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-01-24T18:31:48.457 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:48.627 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:48.896 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:49.354 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:49.354 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:03:45 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:49.806 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:31:49.806 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:45 ago on Tue 24 Jan 2023 06:28:04 PM UTC. 2023-01-24T18:31:51.220 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-01-24T18:31:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:51.318 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-01-24T18:31:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:51.319 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-01-24T18:31:51.442 DEBUG:teuthology.parallel:result is None 2023-01-24T18:31:51.862 INFO:teuthology.orchestra.run.smithi169.stdout:Installed products updated. 2023-01-24T18:31:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:51.959 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-01-24T18:31:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:31:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:51.960 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-01-24T18:31:52.071 DEBUG:teuthology.parallel:result is None 2023-01-24T18:31:52.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo lsb_release -is 2023-01-24T18:31:52.109 INFO:teuthology.orchestra.run.smithi169.stdout:RedHatEnterprise 2023-01-24T18:31:52.111 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:31:52.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:31:52.275 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T18:31:52.293 INFO:teuthology.orchestra.run.smithi169.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T18:31:52.294 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T18:31:52.294 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T18:31:52.295 DEBUG:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2023-01-24T18:31:52.340 INFO:teuthology.orchestra.run.smithi186.stdout:RedHatEnterprise 2023-01-24T18:31:52.341 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:31:52.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:31:52.490 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T18:31:52.511 INFO:teuthology.orchestra.run.smithi186.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T18:31:52.512 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T18:31:52.512 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T18:31:52.512 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T18:31:52.513 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:52.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T18:31:52.536 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:52.537 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T18:31:52.577 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T18:31:52.578 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:52.578 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T18:31:52.601 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T18:31:52.665 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:52.666 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T18:31:52.692 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T18:31:52.757 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T18:31:52.758 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:52.758 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T18:31:52.781 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T18:31:52.844 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:52.844 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T18:31:52.869 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T18:31:52.933 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T18:31:52.999 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T18:31:53.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2023-01-24T18:31:53.002 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2023-01-24T18:31:53.026 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T18:31:53.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T18:31:53.066 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T18:31:53.090 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T18:31:53.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T18:31:53.129 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T18:31:53.165 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T18:31:53.165 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T18:31:53.166 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135589', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135589', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi169.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Qg8vdWmitOAr1GSJyROpi3BklFzbLUQDd9fO0REOvEhywBNzNewiDi9iMmyIOKnz6nW2LapuKifLXw4MchFEi+bVfSKZ2PhHlylj2qOj5CBAzxGR+2yNrTmEveoWayZuqKqIoLQ1CpAtPIIKlOfCXs+ycP+cz3NhICtgswpuhOJ9nNMKg4aD+1IT0jeRAW/FVvNbPngrbXxF1uzfMFTWoIw0ylhGItPCZBY2O6iKMKE15tdA/q5sU6VHxAYNZvLhzYNFxhBfT55Ux4vBqFW32FLFlmAa0qP48dN0hIpACDJEai0ZlxoIpIKhocq6xZqhT3GUAKXmhMm5djV6VrtjUvytNMYbkMr2P3VkD7aRwVymhF+TGokcckaBWoE5MmF0LtHYBLHD6W1MaxgV8SVwfAFey6B56jZGRii6gGkkEZVT0PzxRnGAX6Ik5o6vl4lUaNrzyv/+BjFmWNFSkEEgdZdILlC7xYxxoyIYh8kI5YFC5vL3cEVRwHLqvE/nIS8=', 'smithi186.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxp/U1GuW4XESvFD2xHUvyhjJcR+j5BbtoSayqe23c/pGRx0Bnuj83ynTzvhfbsbW8vVoBizOMp9BaSTRSHUSi+VCFTEjNk/qsUNQlPIXDS+hgkbzVFKxT7T1HB7TLRUIFTRJIF0JNCqyMLf+F/2POxhUBwV5ksWSJD4CqownVJJnhNcn2xwu+7OwDR0SOlEVnatjnhNTiUyCC54yFfYqQl/QTBjEvQV0CyFAecAbD1YjPmphNH690rHVVIykDpVE0MUgIfDm4JJ9jIJLqZKDZrf7+jn39vfc5bdbxT2pd/4euRkDsJBycy0q+QupqUdZtlMXbeA686dMjo0gv+/erHJ7XK3jSA434N3iS7bDJUT7i4VrxWty4UuQIPjs7YgNGygwDe0aaNjjUchAs+ivHBr5fzNGeOgNUNPybgDjgBXIMekwh1sfWhF31M5+Zx1cqcE5jXIuDOSLKWvGFUePtXl6zWvn/B8Ve58G+aVspWXhGANgQqHnwT9D9N1z+3/E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache readproxy', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T18:31:53.166 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T18:31:53.192 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T18:31:53.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T18:31:53.246 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T18:31:53.270 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:53.270 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T18:31:53.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T18:31:53.322 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2023-01-24T18:31:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T18:31:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 201098 Links: 1 2023-01-24T18:31:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-01-24 18:31:41.849965699 +0000 2023-01-24T18:31:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-01-24 18:27:21.261773293 +0000 2023-01-24T18:31:53.377 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-01-24 18:27:21.261773293 +0000 2023-01-24T18:31:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-01-24T18:31:53.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T18:31:53.437 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-01-24T18:31:53.437 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-01-24T18:31:53.438 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00023725 s, 2.2 MB/s 2023-01-24T18:31:53.439 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T18:31:53.494 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2023-01-24T18:31:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T18:31:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:53.548 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 201207 Links: 1 2023-01-24T18:31:53.549 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:53.549 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:53.549 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-01-24 18:31:41.849965699 +0000 2023-01-24T18:31:53.549 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-01-24 18:27:21.765763213 +0000 2023-01-24T18:31:53.549 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-01-24 18:27:21.765763213 +0000 2023-01-24T18:31:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-01-24T18:31:53.550 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T18:31:53.610 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-01-24T18:31:53.610 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-01-24T18:31:53.610 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000193836 s, 2.6 MB/s 2023-01-24T18:31:53.611 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T18:31:53.667 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2023-01-24T18:31:53.721 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T18:31:53.722 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:53.722 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 205861 Links: 1 2023-01-24T18:31:53.722 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:53.722 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:53.722 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-01-24 18:31:41.849965699 +0000 2023-01-24T18:31:53.723 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-01-24 18:27:22.252753467 +0000 2023-01-24T18:31:53.723 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-01-24 18:27:22.252753467 +0000 2023-01-24T18:31:53.723 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-01-24T18:31:53.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T18:31:53.783 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-01-24T18:31:53.784 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-01-24T18:31:53.784 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000234955 s, 2.2 MB/s 2023-01-24T18:31:53.785 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T18:31:53.840 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2023-01-24T18:31:53.894 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T18:31:53.894 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:53.895 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 200690 Links: 1 2023-01-24T18:31:53.895 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:53.895 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:53.895 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-01-24 18:31:41.850965680 +0000 2023-01-24T18:31:53.896 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-01-24 18:27:22.739743726 +0000 2023-01-24T18:31:53.896 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-01-24 18:27:22.739743726 +0000 2023-01-24T18:31:53.896 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-01-24T18:31:53.896 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T18:31:53.955 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-01-24T18:31:53.956 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-01-24T18:31:53.956 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00019418 s, 2.6 MB/s 2023-01-24T18:31:53.957 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T18:31:54.012 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:54.012 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:54.013 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T18:31:54.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T18:31:54.030 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-01-24T18:31:54.085 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T18:31:54.086 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:54.086 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 201304 Links: 1 2023-01-24T18:31:54.086 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:54.087 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:54.087 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-01-24 18:31:41.292683810 +0000 2023-01-24T18:31:54.087 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-01-24 18:27:21.279202395 +0000 2023-01-24T18:31:54.087 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-01-24 18:27:21.279202395 +0000 2023-01-24T18:31:54.087 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-01-24T18:31:54.088 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T18:31:54.150 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-01-24T18:31:54.150 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-01-24T18:31:54.150 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000257349 s, 2.0 MB/s 2023-01-24T18:31:54.151 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T18:31:54.208 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-01-24T18:31:54.264 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T18:31:54.264 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:54.264 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 200179 Links: 1 2023-01-24T18:31:54.265 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:54.265 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:54.265 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-01-24 18:31:41.292683810 +0000 2023-01-24T18:31:54.265 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-01-24 18:27:21.781193097 +0000 2023-01-24T18:31:54.265 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-01-24 18:27:21.781193097 +0000 2023-01-24T18:31:54.266 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-01-24T18:31:54.266 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T18:31:54.327 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-01-24T18:31:54.328 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-01-24T18:31:54.328 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000236372 s, 2.2 MB/s 2023-01-24T18:31:54.329 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T18:31:54.385 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-01-24T18:31:54.440 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T18:31:54.441 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:54.441 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 202183 Links: 1 2023-01-24T18:31:54.441 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:54.441 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:54.442 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-01-24 18:31:41.292683810 +0000 2023-01-24T18:31:54.442 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-01-24 18:27:22.255184316 +0000 2023-01-24T18:31:54.442 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-01-24 18:27:22.255184316 +0000 2023-01-24T18:31:54.442 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-01-24T18:31:54.443 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T18:31:54.503 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-01-24T18:31:54.503 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-01-24T18:31:54.504 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000206951 s, 2.5 MB/s 2023-01-24T18:31:54.505 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T18:31:54.561 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-01-24T18:31:54.616 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T18:31:54.616 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:31:54.616 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 203305 Links: 1 2023-01-24T18:31:54.616 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:31:54.617 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:31:54.617 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-01-24 18:31:41.292683810 +0000 2023-01-24T18:31:54.617 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-01-24 18:27:22.763174907 +0000 2023-01-24T18:31:54.617 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-01-24 18:27:22.763174907 +0000 2023-01-24T18:31:54.617 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-01-24T18:31:54.618 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T18:31:54.678 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-01-24T18:31:54.679 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-01-24T18:31:54.679 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00021209 s, 2.4 MB/s 2023-01-24T18:31:54.680 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T18:31:54.735 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:54.736 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:31:54.736 INFO:tasks.ceph:Generating config... 2023-01-24T18:31:54.738 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T18:31:54.738 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T18:31:54.738 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2023-01-24T18:31:54.738 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-24T18:31:54.739 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T18:31:54.739 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T18:31:54.739 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T18:31:54.740 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T18:31:54.740 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T18:31:54.740 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T18:31:54.740 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T18:31:54.741 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T18:31:54.741 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T18:31:54.741 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T18:31:54.741 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-24T18:31:54.742 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-01-24T18:31:54.742 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T18:31:54.742 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T18:31:54.743 INFO:tasks.ceph:[osd] osd delete sleep = 1 2023-01-24T18:31:54.743 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T18:31:54.743 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-01-24T18:31:54.743 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-24T18:31:54.744 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T18:31:54.744 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T18:31:54.744 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T18:31:54.744 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T18:31:54.745 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-24T18:31:54.745 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-24T18:31:54.745 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T18:31:54.745 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-01-24T18:31:54.746 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-24T18:31:54.746 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-24T18:31:54.746 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-24T18:31:54.747 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-24T18:31:54.747 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-24T18:31:54.747 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-24T18:31:54.747 INFO:tasks.ceph:[global] ms inject delay max = 1 2023-01-24T18:31:54.748 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2023-01-24T18:31:54.748 INFO:tasks.ceph:[global] ms inject delay type = osd 2023-01-24T18:31:54.748 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2023-01-24T18:31:54.748 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-24T18:31:54.749 INFO:tasks.ceph:[global] ms type = async 2023-01-24T18:31:54.749 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-24T18:31:54.749 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-24T18:31:54.750 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-24T18:31:54.750 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-24T18:31:54.750 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-24T18:31:54.750 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T18:31:54.751 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T18:31:54.751 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T18:31:54.751 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T18:31:54.751 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T18:31:54.752 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-24T18:31:54.752 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-24T18:31:54.752 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-24T18:31:54.752 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-24T18:31:54.753 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T18:31:54.753 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-24T18:31:54.753 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T18:31:54.754 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T18:31:54.804 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T18:31:54.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T18:31:54.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:31:54.907 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]'), ('mon.b', '172.21.15.186')] 2023-01-24T18:31:54.908 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790],172.21.15.186', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T18:31:54.909 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:54.909 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:31:54.961 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --add b 172.21.15.186 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:31:55.043 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:31:55.043 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: generated fsid 51ff070a-6967-4dc4-8c3c-d7750347282f 2023-01-24T18:31:55.043 INFO:teuthology.orchestra.run.smithi169.stdout:setting min_mon_release = octopus 2023-01-24T18:31:55.043 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 0 2023-01-24T18:31:55.044 INFO:teuthology.orchestra.run.smithi169.stdout:fsid 51ff070a-6967-4dc4-8c3c-d7750347282f 2023-01-24T18:31:55.044 INFO:teuthology.orchestra.run.smithi169.stdout:last_changed 2023-01-24T18:31:55.042435+0000 2023-01-24T18:31:55.044 INFO:teuthology.orchestra.run.smithi169.stdout:created 2023-01-24T18:31:55.042435+0000 2023-01-24T18:31:55.044 INFO:teuthology.orchestra.run.smithi169.stdout:min_mon_release 15 (octopus) 2023-01-24T18:31:55.045 INFO:teuthology.orchestra.run.smithi169.stdout:election_strategy: 3 2023-01-24T18:31:55.045 INFO:teuthology.orchestra.run.smithi169.stdout:0: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.a 2023-01-24T18:31:55.045 INFO:teuthology.orchestra.run.smithi169.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2023-01-24T18:31:55.045 INFO:teuthology.orchestra.run.smithi169.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2023-01-24T18:31:55.045 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T18:31:55.047 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:31:55.101 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 51ff070a-6967-4dc4-8c3c-d7750347282f... 2023-01-24T18:31:55.102 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T18:31:55.143 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T18:31:55.182 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2023-01-24T18:31:55.182 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2023-01-24T18:31:55.182 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T18:31:55.182 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2023-01-24T18:31:55.183 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.183 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2023-01-24T18:31:55.183 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.183 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2023-01-24T18:31:55.183 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.184 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2023-01-24T18:31:55.184 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2023-01-24T18:31:55.184 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.184 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2023-01-24T18:31:55.184 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2023-01-24T18:31:55.185 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.185 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T18:31:55.185 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2023-01-24T18:31:55.185 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.185 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2023-01-24T18:31:55.186 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2023-01-24T18:31:55.187 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2023-01-24T18:31:55.187 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2023-01-24T18:31:55.187 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2023-01-24T18:31:55.187 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2023-01-24T18:31:55.187 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2023-01-24T18:31:55.188 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T18:31:55.188 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T18:31:55.188 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2023-01-24T18:31:55.188 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.188 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T18:31:55.189 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.189 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2023-01-24T18:31:55.189 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.189 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2023-01-24T18:31:55.189 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.190 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2023-01-24T18:31:55.190 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2023-01-24T18:31:55.190 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2023-01-24T18:31:55.190 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790],172.21.15.186" 2023-01-24T18:31:55.190 INFO:teuthology.orchestra.run.smithi169.stdout: mon client directed command retry = 5 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: mon election default strategy = 3 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: ms bind msgr1 = False 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: ms bind msgr2 = True 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: ms inject delay max = 1 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: ms inject delay probability = 0.005 2023-01-24T18:31:55.191 INFO:teuthology.orchestra.run.smithi169.stdout: ms inject delay type = osd 2023-01-24T18:31:55.192 INFO:teuthology.orchestra.run.smithi169.stdout: ms inject internal delays = 0.002 2023-01-24T18:31:55.192 INFO:teuthology.orchestra.run.smithi169.stdout: ms inject socket failures = 2500 2023-01-24T18:31:55.192 INFO:teuthology.orchestra.run.smithi169.stdout: ms type = async 2023-01-24T18:31:55.192 INFO:teuthology.orchestra.run.smithi169.stdout: osd_max_pg_log_entries = 2 2023-01-24T18:31:55.193 INFO:teuthology.orchestra.run.smithi169.stdout: osd_min_pg_log_entries = 1 2023-01-24T18:31:55.193 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pg_log_trim_min = 0 2023-01-24T18:31:55.193 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_min_size = 2 2023-01-24T18:31:55.193 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_size = 3 2023-01-24T18:31:55.193 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 51ff070a-6967-4dc4-8c3c-d7750347282f 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2023-01-24T18:31:55.194 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 120 2023-01-24T18:31:55.195 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.195 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2023-01-24T18:31:55.195 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2023-01-24T18:31:55.195 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.195 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2023-01-24T18:31:55.196 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2023-01-24T18:31:55.196 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2023-01-24T18:31:55.196 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.196 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2023-01-24T18:31:55.196 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2023-01-24T18:31:55.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.197 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2023-01-24T18:31:55.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.197 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2023-01-24T18:31:55.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.198 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2023-01-24T18:31:55.198 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.198 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2023-01-24T18:31:55.198 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2023-01-24T18:31:55.198 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: debug monc = 20 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore compression algorithm = lz4 2023-01-24T18:31:55.199 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore compression mode = aggressive 2023-01-24T18:31:55.200 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2023-01-24T18:31:55.200 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore zero block detection = True 2023-01-24T18:31:55.200 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 20 2023-01-24T18:31:55.200 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 20 2023-01-24T18:31:55.200 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-01-24T18:31:55.201 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2023-01-24T18:31:55.201 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 10 2023-01-24T18:31:55.201 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T18:31:55.201 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2023-01-24T18:31:55.201 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2023-01-24T18:31:55.202 INFO:teuthology.orchestra.run.smithi169.stdout: osd backoff on peering = True 2023-01-24T18:31:55.202 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug reject backfill probability = 0.3 2023-01-24T18:31:55.202 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify cached snaps = True 2023-01-24T18:31:55.202 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify missing on start = True 2023-01-24T18:31:55.202 INFO:teuthology.orchestra.run.smithi169.stdout: osd delete sleep = 1 2023-01-24T18:31:55.203 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2023-01-24T18:31:55.203 INFO:teuthology.orchestra.run.smithi169.stdout: osd max backfills = 3 2023-01-24T18:31:55.203 INFO:teuthology.orchestra.run.smithi169.stdout: osd max markdown count = 1000 2023-01-24T18:31:55.203 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock override recovery settings = True 2023-01-24T18:31:55.203 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2023-01-24T18:31:55.204 INFO:teuthology.orchestra.run.smithi169.stdout: osd op queue = debug_random 2023-01-24T18:31:55.204 INFO:teuthology.orchestra.run.smithi169.stdout: osd op queue cut off = debug_random 2023-01-24T18:31:55.204 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub min interval = 60 2023-01-24T18:31:55.204 INFO:teuthology.orchestra.run.smithi169.stdout: osd shutdown pgref assert = True 2023-01-24T18:31:55.204 INFO:teuthology.orchestra.run.smithi169.stdout: osd snap trim sleep = 2 2023-01-24T18:31:55.205 INFO:teuthology.orchestra.run.smithi169.stdout: osd_max_pg_log_entries = 3000 2023-01-24T18:31:55.205 INFO:teuthology.orchestra.run.smithi169.stdout: osd_min_pg_log_entries = 3000 2023-01-24T18:31:55.205 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.205 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2023-01-24T18:31:55.205 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-01-24T18:31:55.206 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2023-01-24T18:31:55.206 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-01-24T18:31:55.206 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-01-24T18:31:55.206 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:31:55.206 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-01-24T18:31:55.207 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2023-01-24T18:31:55.208 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-01-24T18:31:55.208 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2023-01-24T18:31:55.208 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2023-01-24T18:31:55.208 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:31:55.209 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2023-01-24T18:31:55.209 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2023-01-24T18:31:55.209 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:31:55.209 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.209 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T18:31:55.210 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T18:31:55.210 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T18:31:55.210 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.210 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T18:31:55.211 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T18:31:55.211 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T18:31:55.211 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.211 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2023-01-24T18:31:55.211 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2023-01-24T18:31:55.212 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.212 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2023-01-24T18:31:55.212 INFO:teuthology.orchestra.run.smithi169.stdout: mon min osdmap epochs = 50 2023-01-24T18:31:55.212 INFO:teuthology.orchestra.run.smithi169.stdout: mon osdmap full prune interval = 2 2023-01-24T18:31:55.212 INFO:teuthology.orchestra.run.smithi169.stdout: mon osdmap full prune min = 15 2023-01-24T18:31:55.213 INFO:teuthology.orchestra.run.smithi169.stdout: mon osdmap full prune txsize = 2 2023-01-24T18:31:55.213 INFO:teuthology.orchestra.run.smithi169.stdout: mon scrub interval = 300 2023-01-24T18:31:55.213 INFO:teuthology.orchestra.run.smithi169.stdout: paxos service trim min = 10 2023-01-24T18:31:55.213 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:31:55.213 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2023-01-24T18:31:55.214 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2023-01-24T18:31:55.214 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2023-01-24T18:31:55.214 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2023-01-24T18:31:55.214 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T18:31:55.214 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T18:31:55.215 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2023-01-24T18:31:55.215 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2023-01-24T18:31:55.215 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2023-01-24T18:31:55.216 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2023-01-24T18:31:55.217 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2023-01-24T18:31:55.217 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T18:31:55.217 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2023-01-24T18:31:55.217 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.218 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2023-01-24T18:31:55.218 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.218 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2023-01-24T18:31:55.218 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.218 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2023-01-24T18:31:55.219 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2023-01-24T18:31:55.219 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.219 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2023-01-24T18:31:55.219 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2023-01-24T18:31:55.219 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T18:31:55.220 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2023-01-24T18:31:55.220 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.220 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T18:31:55.220 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T18:31:55.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.221 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2023-01-24T18:31:55.221 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.221 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2023-01-24T18:31:55.221 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2023-01-24T18:31:55.222 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2023-01-24T18:31:55.222 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2023-01-24T18:31:55.222 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2023-01-24T18:31:55.222 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2023-01-24T18:31:55.222 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2023-01-24T18:31:55.223 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T18:31:55.223 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T18:31:55.223 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2023-01-24T18:31:55.223 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.223 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T18:31:55.224 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.224 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2023-01-24T18:31:55.224 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.224 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2023-01-24T18:31:55.224 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2023-01-24T18:31:55.225 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2023-01-24T18:31:55.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2023-01-24T18:31:55.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790],172.21.15.186" 2023-01-24T18:31:55.225 INFO:teuthology.orchestra.run.smithi186.stdout: mon client directed command retry = 5 2023-01-24T18:31:55.226 INFO:teuthology.orchestra.run.smithi186.stdout: mon election default strategy = 3 2023-01-24T18:31:55.226 INFO:teuthology.orchestra.run.smithi186.stdout: ms bind msgr1 = False 2023-01-24T18:31:55.226 INFO:teuthology.orchestra.run.smithi186.stdout: ms bind msgr2 = True 2023-01-24T18:31:55.226 INFO:teuthology.orchestra.run.smithi186.stdout: ms inject delay max = 1 2023-01-24T18:31:55.226 INFO:teuthology.orchestra.run.smithi186.stdout: ms inject delay probability = 0.005 2023-01-24T18:31:55.227 INFO:teuthology.orchestra.run.smithi186.stdout: ms inject delay type = osd 2023-01-24T18:31:55.227 INFO:teuthology.orchestra.run.smithi186.stdout: ms inject internal delays = 0.002 2023-01-24T18:31:55.227 INFO:teuthology.orchestra.run.smithi186.stdout: ms inject socket failures = 2500 2023-01-24T18:31:55.227 INFO:teuthology.orchestra.run.smithi186.stdout: ms type = async 2023-01-24T18:31:55.227 INFO:teuthology.orchestra.run.smithi186.stdout: osd_max_pg_log_entries = 2 2023-01-24T18:31:55.228 INFO:teuthology.orchestra.run.smithi186.stdout: osd_min_pg_log_entries = 1 2023-01-24T18:31:55.228 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pg_log_trim_min = 0 2023-01-24T18:31:55.228 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_min_size = 2 2023-01-24T18:31:55.228 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_size = 3 2023-01-24T18:31:55.228 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 51ff070a-6967-4dc4-8c3c-d7750347282f 2023-01-24T18:31:55.229 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.229 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2023-01-24T18:31:55.229 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2023-01-24T18:31:55.229 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.229 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 120 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.230 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2023-01-24T18:31:55.231 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2023-01-24T18:31:55.231 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2023-01-24T18:31:55.231 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.231 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2023-01-24T18:31:55.231 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2023-01-24T18:31:55.232 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.232 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2023-01-24T18:31:55.232 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.232 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2023-01-24T18:31:55.232 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.233 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2023-01-24T18:31:55.233 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.233 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2023-01-24T18:31:55.233 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2023-01-24T18:31:55.233 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.234 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2023-01-24T18:31:55.234 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2023-01-24T18:31:55.234 INFO:teuthology.orchestra.run.smithi186.stdout: debug monc = 20 2023-01-24T18:31:55.234 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2023-01-24T18:31:55.234 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore compression algorithm = lz4 2023-01-24T18:31:55.235 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore compression mode = aggressive 2023-01-24T18:31:55.235 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2023-01-24T18:31:55.235 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore zero block detection = True 2023-01-24T18:31:55.235 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 20 2023-01-24T18:31:55.235 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 20 2023-01-24T18:31:55.236 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-01-24T18:31:55.236 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2023-01-24T18:31:55.236 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 10 2023-01-24T18:31:55.236 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T18:31:55.236 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2023-01-24T18:31:55.237 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2023-01-24T18:31:55.237 INFO:teuthology.orchestra.run.smithi186.stdout: osd backoff on peering = True 2023-01-24T18:31:55.237 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug reject backfill probability = 0.3 2023-01-24T18:31:55.237 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify cached snaps = True 2023-01-24T18:31:55.237 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify missing on start = True 2023-01-24T18:31:55.238 INFO:teuthology.orchestra.run.smithi186.stdout: osd delete sleep = 1 2023-01-24T18:31:55.238 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2023-01-24T18:31:55.238 INFO:teuthology.orchestra.run.smithi186.stdout: osd max backfills = 3 2023-01-24T18:31:55.238 INFO:teuthology.orchestra.run.smithi186.stdout: osd max markdown count = 1000 2023-01-24T18:31:55.238 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock override recovery settings = True 2023-01-24T18:31:55.239 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2023-01-24T18:31:55.239 INFO:teuthology.orchestra.run.smithi186.stdout: osd op queue = debug_random 2023-01-24T18:31:55.239 INFO:teuthology.orchestra.run.smithi186.stdout: osd op queue cut off = debug_random 2023-01-24T18:31:55.239 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub min interval = 60 2023-01-24T18:31:55.239 INFO:teuthology.orchestra.run.smithi186.stdout: osd shutdown pgref assert = True 2023-01-24T18:31:55.240 INFO:teuthology.orchestra.run.smithi186.stdout: osd snap trim sleep = 2 2023-01-24T18:31:55.240 INFO:teuthology.orchestra.run.smithi186.stdout: osd_max_pg_log_entries = 3000 2023-01-24T18:31:55.240 INFO:teuthology.orchestra.run.smithi186.stdout: osd_min_pg_log_entries = 3000 2023-01-24T18:31:55.240 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.240 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2023-01-24T18:31:55.241 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-01-24T18:31:55.241 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2023-01-24T18:31:55.241 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-01-24T18:31:55.241 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-01-24T18:31:55.241 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:31:55.242 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:31:55.242 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2023-01-24T18:31:55.242 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.242 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2023-01-24T18:31:55.242 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2023-01-24T18:31:55.243 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:31:55.244 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2023-01-24T18:31:55.244 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2023-01-24T18:31:55.244 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:31:55.244 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.244 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T18:31:55.245 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T18:31:55.245 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T18:31:55.245 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.245 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T18:31:55.245 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T18:31:55.246 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T18:31:55.246 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.246 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2023-01-24T18:31:55.246 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2023-01-24T18:31:55.246 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon min osdmap epochs = 50 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon osdmap full prune interval = 2 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon osdmap full prune min = 15 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon osdmap full prune txsize = 2 2023-01-24T18:31:55.247 INFO:teuthology.orchestra.run.smithi186.stdout: mon scrub interval = 300 2023-01-24T18:31:55.248 INFO:teuthology.orchestra.run.smithi186.stdout: paxos service trim min = 10 2023-01-24T18:31:55.248 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-01-24T18:31:55.248 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2023-01-24T18:31:55.248 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2023-01-24T18:31:55.248 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2023-01-24T18:31:55.249 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2023-01-24T18:31:55.249 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T18:31:55.249 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T18:31:55.249 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2023-01-24T18:31:55.249 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2023-01-24T18:31:55.250 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2023-01-24T18:31:55.250 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T18:31:55.250 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T18:31:55.306 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T18:31:55.306 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:55.307 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T18:31:55.321 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:55.321 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T18:31:55.375 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:31:55.376 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:55.376 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T18:31:55.376 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:31:55.448 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:31:55.448 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:31:55.502 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:31:55.502 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:55.503 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T18:31:55.503 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:31:55.537 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:31:55.537 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:31:55.591 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T18:31:55.592 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T18:31:55.592 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T18:31:55.639 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T18:31:55.641 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:31:55.702 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:31:55.704 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T18:31:55.705 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T18:31:55.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T18:31:55.742 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T18:31:55.754 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T18:31:55.790 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T18:31:55.802 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T18:31:55.803 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:31:55.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T18:31:55.826 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:55.826 INFO:tasks.ceph:role: osd.0 2023-01-24T18:31:55.827 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:31:55.827 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T18:31:55.897 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:31:55.897 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:31:55.897 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:31:55.897 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-01-24T18:31:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:31:55.898 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:31:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:31:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:31:55.898 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:31:55.899 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:31:56.261 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-01-24T18:31:56.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-01-24T18:31:56.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T18:31:56.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T18:31:56.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T18:31:56.469 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:56.470 INFO:tasks.ceph:role: osd.1 2023-01-24T18:31:56.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:31:56.471 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T18:31:56.540 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:31:56.540 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:31:56.540 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:31:56.541 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-01-24T18:31:56.541 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:31:56.541 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:31:56.541 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:31:56.542 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:31:56.542 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:31:56.542 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:31:56.907 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-01-24T18:31:56.909 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-01-24T18:31:56.909 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T18:31:56.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T18:31:57.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T18:31:57.088 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:57.088 INFO:tasks.ceph:role: osd.2 2023-01-24T18:31:57.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:31:57.089 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T18:31:57.158 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:31:57.158 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:31:57.159 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:31:57.159 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-01-24T18:31:57.159 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:31:57.159 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:31:57.159 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:31:57.160 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:31:57.160 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:31:57.160 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:31:57.526 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-01-24T18:31:57.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-01-24T18:31:57.528 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T18:31:57.640 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T18:31:57.667 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T18:31:57.731 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:31:57.731 INFO:tasks.ceph:role: osd.3 2023-01-24T18:31:57.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2023-01-24T18:31:57.732 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T18:31:57.802 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:31:57.803 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:31:57.803 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:31:57.803 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-01-24T18:31:57.803 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:31:57.803 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:31:57.804 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:31:57.804 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:31:57.804 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:31:57.804 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:31:58.177 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-01-24T18:31:58.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-01-24T18:31:58.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T18:31:58.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T18:31:58.312 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:31:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.390+0000 7fb936635540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T18:31:58.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.391+0000 7fb936635540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T18:31:58.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.391+0000 7fb936635540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:31:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.391+0000 7fb936635540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:31:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.391+0000 7fb936635540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:31:58.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:31:58.396+0000 7fb936635540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T18:32:01.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T18:32:01.320 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:01.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T18:32:01.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T18:32:01.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:32:01.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:32:01.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:32:01.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:01.403+0000 7fc21dc8c540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T18:32:04.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T18:32:04.331 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:04.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.412+0000 7f742f87f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T18:32:04.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.412+0000 7f742f87f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T18:32:04.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.412+0000 7f742f87f540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:32:04.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.412+0000 7f742f87f540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:32:04.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.412+0000 7f742f87f540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:32:04.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:04.413+0000 7f742f87f540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T18:32:07.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T18:32:07.340 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:07.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.423+0000 7f6d4d3a2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T18:32:07.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.423+0000 7f6d4d3a2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T18:32:07.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.423+0000 7f6d4d3a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:32:07.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.423+0000 7f6d4d3a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:32:07.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.423+0000 7f6d4d3a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:32:07.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-01-24T18:32:07.424+0000 7f6d4d3a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T18:32:10.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T18:32:10.351 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T18:32:10.380 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:32:10.380 INFO:tasks.ceph:role: osd.4 2023-01-24T18:32:10.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:32:10.381 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T18:32:10.455 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:32:10.455 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:32:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:32:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 2023-01-24T18:32:10.456 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:32:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:32:10.456 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:32:10.457 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:32:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:32:10.457 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:32:10.823 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-01-24T18:32:10.825 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-01-24T18:32:10.825 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T18:32:10.957 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T18:32:10.988 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T18:32:11.055 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:32:11.055 INFO:tasks.ceph:role: osd.5 2023-01-24T18:32:11.055 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:32:11.056 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T18:32:11.128 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:32:11.128 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:32:11.128 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:32:11.129 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 2023-01-24T18:32:11.129 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:32:11.129 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:32:11.129 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:32:11.130 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:32:11.130 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:32:11.130 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:32:11.493 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-01-24T18:32:11.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-01-24T18:32:11.495 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T18:32:11.579 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T18:32:11.647 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T18:32:11.711 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:32:11.711 INFO:tasks.ceph:role: osd.6 2023-01-24T18:32:11.712 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:32:11.712 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T18:32:11.782 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:32:11.783 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:32:11.783 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:32:11.783 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 2023-01-24T18:32:11.783 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:32:11.783 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:32:11.784 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:32:11.784 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:32:11.784 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:32:11.784 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:32:12.150 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-01-24T18:32:12.152 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-01-24T18:32:12.152 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T18:32:12.256 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T18:32:12.284 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T18:32:12.348 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T18:32:12.349 INFO:tasks.ceph:role: osd.7 2023-01-24T18:32:12.349 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2023-01-24T18:32:12.349 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T18:32:12.420 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:32:12.420 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:32:12.420 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:32:12.421 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 2023-01-24T18:32:12.421 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:32:12.421 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:32:12.421 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:32:12.421 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:32:12.422 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:32:12.422 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:32:12.796 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-01-24T18:32:12.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-01-24T18:32:12.798 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T18:32:12.912 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T18:32:12.939 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:13.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.019+0000 7f73ae879540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T18:32:13.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.021+0000 7f73ae879540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T18:32:13.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.021+0000 7f73ae879540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:32:13.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.021+0000 7f73ae879540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:32:13.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.022+0000 7f73ae879540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:32:13.027 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:13.026+0000 7f73ae879540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T18:32:15.881 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T18:32:15.951 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:16.037 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T18:32:16.037 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T18:32:16.037 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:32:16.038 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:32:16.038 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:32:16.038 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:16.036+0000 7ff558311540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T18:32:18.890 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T18:32:18.919 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:19.003 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.001+0000 7fe8cf436540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T18:32:19.003 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.001+0000 7fe8cf436540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T18:32:19.004 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.001+0000 7fe8cf436540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:32:19.004 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.001+0000 7fe8cf436540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:32:19.004 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.001+0000 7fe8cf436540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:32:19.004 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:19.002+0000 7fe8cf436540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T18:32:21.865 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T18:32:21.934 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:22.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.017+0000 7f7a4dae3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T18:32:22.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.017+0000 7f7a4dae3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T18:32:22.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.017+0000 7f7a4dae3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:32:22.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.017+0000 7f7a4dae3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:32:22.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.017+0000 7f7a4dae3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:32:22.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-01-24T18:32:22.018+0000 7f7a4dae3540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T18:32:24.883 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T18:32:24.912 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T18:32:24.912 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:24.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T18:32:24.941 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:24.942 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T18:32:25.007 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:25.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T18:32:25.074 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:25.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T18:32:25.139 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:25.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T18:32:25.205 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T18:32:25.235 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.236 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T18:32:25.302 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.303 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T18:32:25.369 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.369 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T18:32:25.435 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.436 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T18:32:25.500 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:25.501 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T18:32:25.517 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:25.517 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T18:32:25.555 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T18:32:25.556 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T18:32:25.559 DEBUG:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T18:32:25.598 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2023-01-24T18:32:25.598 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAbJNBj+t8PJhAAKk5XbaURr5nyiK+gU4740w== 2023-01-24T18:32:25.598 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2023-01-24T18:32:25.599 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAeJNBjLHtQFxAAu3KRO0VLvGR0WTo7AfvMzw== 2023-01-24T18:32:25.599 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2023-01-24T18:32:25.599 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAhJNBjfNcTGBAAuc4KelRz48ZRP/PNw2oHXg== 2023-01-24T18:32:25.599 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2023-01-24T18:32:25.599 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAkJNBjbVygGBAAM/pmRxva0CeJcL4YkXxUiQ== 2023-01-24T18:32:25.600 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2023-01-24T18:32:25.600 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAnJNBjTW1JGRAACWUZBTrOPJg9+HMc1eJbxQ== 2023-01-24T18:32:25.600 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2023-01-24T18:32:25.600 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAbJNBjBoTZKRAAJ+ErkrP6FpLGcbeRcR1J+Q== 2023-01-24T18:32:25.600 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2023-01-24T18:32:25.601 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAtJNBjccglARAAUGH9Se/iVLKfOzNHwnCiUQ== 2023-01-24T18:32:25.601 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2023-01-24T18:32:25.601 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAwJNBjNiArAhAAMmak44IvMPnmC8U5XiuWwg== 2023-01-24T18:32:25.601 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2023-01-24T18:32:25.601 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAzJNBjuVIpABAAquS/Nbfzw3lahERry74TLg== 2023-01-24T18:32:25.602 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2023-01-24T18:32:25.602 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA2JNBjZFgfARAAfHTJutvnA4qBNMGGUJft6w== 2023-01-24T18:32:25.602 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-01-24T18:32:25.602 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAbJNBj81w2LBAAF2kfpMRKhBdbOPSGbhszww== 2023-01-24T18:32:25.602 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-01-24T18:32:25.603 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAbJNBjbq0ZLxAA/Sgr5+tlUFFerMovOmx71g== 2023-01-24T18:32:25.618 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2023-01-24T18:32:25.618 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAbJNBj+t8PJhAAKk5XbaURr5nyiK+gU4740w== 2023-01-24T18:32:25.619 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2023-01-24T18:32:25.619 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAeJNBjLHtQFxAAu3KRO0VLvGR0WTo7AfvMzw== 2023-01-24T18:32:25.619 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2023-01-24T18:32:25.619 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAhJNBjfNcTGBAAuc4KelRz48ZRP/PNw2oHXg== 2023-01-24T18:32:25.619 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2023-01-24T18:32:25.620 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAkJNBjbVygGBAAM/pmRxva0CeJcL4YkXxUiQ== 2023-01-24T18:32:25.620 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2023-01-24T18:32:25.620 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAnJNBjTW1JGRAACWUZBTrOPJg9+HMc1eJbxQ== 2023-01-24T18:32:25.620 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2023-01-24T18:32:25.620 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAbJNBjBoTZKRAAJ+ErkrP6FpLGcbeRcR1J+Q== 2023-01-24T18:32:25.621 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2023-01-24T18:32:25.621 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAtJNBjccglARAAUGH9Se/iVLKfOzNHwnCiUQ== 2023-01-24T18:32:25.621 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2023-01-24T18:32:25.621 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAwJNBjNiArAhAAMmak44IvMPnmC8U5XiuWwg== 2023-01-24T18:32:25.621 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2023-01-24T18:32:25.622 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAzJNBjuVIpABAAquS/Nbfzw3lahERry74TLg== 2023-01-24T18:32:25.622 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2023-01-24T18:32:25.622 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQA2JNBjZFgfARAAfHTJutvnA4qBNMGGUJft6w== 2023-01-24T18:32:25.622 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2023-01-24T18:32:25.622 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAbJNBj81w2LBAAF2kfpMRKhBdbOPSGbhszww== 2023-01-24T18:32:25.623 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2023-01-24T18:32:25.623 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAbJNBjbq0ZLxAA/Sgr5+tlUFFerMovOmx71g== 2023-01-24T18:32:25.624 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:32:25.646 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:32:25.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.715 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.764 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.809 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.811 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.897 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.943 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:32:25.946 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:32:25.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:25.994 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.042 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.088 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.090 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.139 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:32:26.185 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:32:26.187 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:32:26.235 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:32:26.238 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:32:26.286 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T18:32:26.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T18:32:26.310 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:32:26.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T18:32:26.423 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T18:32:26.486 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:32:26.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T18:32:26.599 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T18:32:26.623 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:32:26.714 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T18:32:26.738 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:26.741 DEBUG:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:32:26.792 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T18:32:26.793 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T18:32:26.793 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T18:32:26.795 INFO:tasks.ceph.mon.a:Started 2023-01-24T18:32:26.796 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T18:32:26.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T18:32:26.798 INFO:tasks.ceph.mon.c:Started 2023-01-24T18:32:26.798 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T18:32:26.799 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T18:32:26.835 INFO:tasks.ceph.mon.b:Started 2023-01-24T18:32:26.835 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T18:32:26.836 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T18:32:26.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T18:32:26.840 INFO:tasks.ceph.mgr.y:Started 2023-01-24T18:32:26.841 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T18:32:26.841 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T18:32:26.843 INFO:tasks.ceph.mgr.x:Started 2023-01-24T18:32:26.843 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T18:32:26.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T18:32:33.241 INFO:teuthology.orchestra.run.smithi169.stderr:adjusted tunables profile to default 2023-01-24T18:32:33.252 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T18:32:33.253 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:33.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T18:32:33.277 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:33.277 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T18:32:33.341 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:33.341 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T18:32:33.407 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-01-24T18:32:33.407 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T18:32:33.470 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:33.471 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T18:32:33.495 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:33.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T18:32:33.561 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:33.561 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T18:32:33.625 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-01-24T18:32:33.625 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T18:32:33.689 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 83346d42-55a8-4fca-aad6-fffb1fb67752 0 2023-01-24T18:32:33.941 INFO:teuthology.orchestra.run.smithi186.stdout:0 2023-01-24T18:32:33.954 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36 1 2023-01-24T18:32:34.171 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-01-24T18:32:34.183 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 4ec36bfe-4e7a-493b-908a-afa16532e172 2 2023-01-24T18:32:34.291 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.288+0000 7fedd4cfa180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T18:32:34.291 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.288+0000 7fedd4cfa180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T18:32:34.292 INFO:tasks.ceph.mgr.y.smithi169.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T18:32:34.292 INFO:tasks.ceph.mgr.y.smithi169.stderr: from .module import Module, StandbyModule 2023-01-24T18:32:34.292 INFO:tasks.ceph.mgr.y.smithi169.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T18:32:34.292 INFO:tasks.ceph.mgr.y.smithi169.stderr: from packaging.version import Version 2023-01-24T18:32:34.292 INFO:tasks.ceph.mgr.y.smithi169.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T18:32:34.293 INFO:tasks.ceph.mgr.y.smithi169.stderr: 2023-01-24T18:32:34.293 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.289+0000 7fedd4cfa180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T18:32:34.293 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.289+0000 7fedd4cfa180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T18:32:34.317 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.315+0000 7f2f924e2180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T18:32:34.317 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.315+0000 7f2f924e2180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T18:32:34.318 INFO:tasks.ceph.mgr.x.smithi186.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T18:32:34.318 INFO:tasks.ceph.mgr.x.smithi186.stderr: from .module import Module, StandbyModule 2023-01-24T18:32:34.318 INFO:tasks.ceph.mgr.x.smithi186.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T18:32:34.318 INFO:tasks.ceph.mgr.x.smithi186.stderr: from packaging.version import Version 2023-01-24T18:32:34.319 INFO:tasks.ceph.mgr.x.smithi186.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T18:32:34.319 INFO:tasks.ceph.mgr.x.smithi186.stderr: 2023-01-24T18:32:34.319 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.316+0000 7f2f924e2180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T18:32:34.319 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.316+0000 7f2f924e2180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T18:32:34.400 INFO:teuthology.orchestra.run.smithi186.stdout:2 2023-01-24T18:32:34.412 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 3c96656a-f484-4701-85d7-eb9fe4bd57a2 3 2023-01-24T18:32:34.422 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.420+0000 7fedd4cfa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T18:32:34.453 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.452+0000 7f2f924e2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T18:32:34.496 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.493+0000 7fedd4cfa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T18:32:34.531 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.530+0000 7f2f924e2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T18:32:34.628 INFO:teuthology.orchestra.run.smithi186.stdout:3 2023-01-24T18:32:34.641 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new d96403b8-122b-443c-8ef0-2806cc1a630d 4 2023-01-24T18:32:34.678 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.676+0000 7fedd4cfa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:32:34.724 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.722+0000 7f2f924e2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:32:34.846 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.844+0000 7fedd4cfa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T18:32:34.892 INFO:teuthology.orchestra.run.smithi186.stdout:4 2023-01-24T18:32:34.901 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.900+0000 7f2f924e2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T18:32:34.904 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new cf3e568a-94cc-46b3-8e48-f44cc56ab781 5 2023-01-24T18:32:34.922 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:34.920+0000 7fedd4cfa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T18:32:34.981 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:34.979+0000 7f2f924e2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T18:32:35.065 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:35.063+0000 7fedd4cfa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T18:32:35.122 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:35.121+0000 7f2f924e2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T18:32:35.160 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:35.158+0000 7fedd4cfa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T18:32:35.160 INFO:teuthology.orchestra.run.smithi186.stdout:5 2023-01-24T18:32:35.172 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 775579af-2d90-46b3-8e7a-0c1d3aaf1e69 6 2023-01-24T18:32:35.219 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:35.217+0000 7f2f924e2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T18:32:35.426 INFO:teuthology.orchestra.run.smithi186.stdout:6 2023-01-24T18:32:35.438 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new e1d5397a-666f-4a76-b897-91e859a2c942 7 2023-01-24T18:32:35.666 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:35.664+0000 7fedd4cfa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T18:32:35.693 INFO:teuthology.orchestra.run.smithi186.stdout:7 2023-01-24T18:32:35.705 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T18:32:35.706 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T18:32:35.708 INFO:tasks.ceph.osd.0:Started 2023-01-24T18:32:35.708 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T18:32:35.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T18:32:35.711 INFO:tasks.ceph.osd.1:Started 2023-01-24T18:32:35.711 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T18:32:35.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T18:32:35.713 INFO:tasks.ceph.osd.2:Started 2023-01-24T18:32:35.714 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T18:32:35.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T18:32:35.716 INFO:tasks.ceph.osd.3:Started 2023-01-24T18:32:35.717 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T18:32:35.717 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T18:32:35.720 INFO:tasks.ceph.osd.4:Started 2023-01-24T18:32:35.720 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T18:32:35.720 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T18:32:35.722 INFO:tasks.ceph.osd.5:Started 2023-01-24T18:32:35.723 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T18:32:35.723 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T18:32:35.725 INFO:tasks.ceph.osd.6:Started 2023-01-24T18:32:35.725 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T18:32:35.726 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T18:32:35.728 INFO:tasks.ceph.osd.7:Started 2023-01-24T18:32:35.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:32:35.758 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:35.756+0000 7fedd4cfa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T18:32:35.760 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:35.759+0000 7f2f924e2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T18:32:35.845 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:35.843+0000 7f2f924e2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T18:32:35.853 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:35.851+0000 7fedd4cfa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T18:32:35.922 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:35.920+0000 7f2f924e2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T18:32:35.946 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:35.946 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":10,"fsid":"51ff070a-6967-4dc4-8c3c-d7750347282f","created":"2023-01-24T18:32:33.111789+0000","modified":"2023-01-24T18:32:35.688856+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:32:35.688856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"83346d42-55a8-4fca-aad6-fffb1fb67752","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4ec36bfe-4e7a-493b-908a-afa16532e172","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3c96656a-f484-4701-85d7-eb9fe4bd57a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d96403b8-122b-443c-8ef0-2806cc1a630d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cf3e568a-94cc-46b3-8e48-f44cc56ab781","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"775579af-2d90-46b3-8e7a-0c1d3aaf1e69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e1d5397a-666f-4a76-b897-91e859a2c942","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:32:35.958 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T18:32:35.959 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T18:32:36.314 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-01-24T18:32:36.312+0000 7fae2ac4a540 -1 Falling back to public interface 2023-01-24T18:32:36.320 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-01-24T18:32:36.318+0000 7f94f1213540 -1 Falling back to public interface 2023-01-24T18:32:36.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-01-24T18:32:36.320+0000 7fc28fb40540 -1 Falling back to public interface 2023-01-24T18:32:36.328 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-01-24T18:32:36.326+0000 7ff428614540 -1 Falling back to public interface 2023-01-24T18:32:36.329 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-01-24T18:32:36.326+0000 7ff422a3c700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2023-01-24T18:32:36.332 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-01-24T18:32:36.326+0000 7efd94000540 -1 Falling back to public interface 2023-01-24T18:32:36.332 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-01-24T18:32:36.330+0000 7f8d6d67b540 -1 Falling back to public interface 2023-01-24T18:32:36.340 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-01-24T18:32:36.338+0000 7f23ad12b540 -1 Falling back to public interface 2023-01-24T18:32:36.341 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-01-24T18:32:36.339+0000 7fc38b048540 -1 Falling back to public interface 2023-01-24T18:32:36.444 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:36.442+0000 7fedd4cfa180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T18:32:36.494 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:36.493+0000 7f2f924e2180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T18:32:36.578 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:36.576+0000 7fedd4cfa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:32:36.631 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:36.629+0000 7f2f924e2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:32:36.674 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:36.671+0000 7fedd4cfa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T18:32:36.722 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:36.721+0000 7f2f924e2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T18:32:36.763 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:32:36.927 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:36.925+0000 7fedd4cfa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T18:32:36.956 INFO:teuthology.misc.health.smithi169.stdout: 2023-01-24T18:32:36.957 INFO:teuthology.misc.health.smithi169.stdout:{"epoch":10,"fsid":"51ff070a-6967-4dc4-8c3c-d7750347282f","created":"2023-01-24T18:32:33.111789+0000","modified":"2023-01-24T18:32:35.688856+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:32:35.688856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"83346d42-55a8-4fca-aad6-fffb1fb67752","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4ec36bfe-4e7a-493b-908a-afa16532e172","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3c96656a-f484-4701-85d7-eb9fe4bd57a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d96403b8-122b-443c-8ef0-2806cc1a630d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cf3e568a-94cc-46b3-8e48-f44cc56ab781","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"775579af-2d90-46b3-8e7a-0c1d3aaf1e69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e1d5397a-666f-4a76-b897-91e859a2c942","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:32:36.967 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T18:32:36.970 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:36.969+0000 7f2f924e2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T18:32:37.005 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:37.003+0000 7fedd4cfa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T18:32:37.047 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:37.045+0000 7f2f924e2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T18:32:37.581 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:37.579+0000 7fedd4cfa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T18:32:37.619 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:37.618+0000 7f2f924e2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T18:32:37.824 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:37.821+0000 7fedd4cfa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T18:32:37.862 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:37.860+0000 7f2f924e2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T18:32:37.955 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:37.952+0000 7fedd4cfa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T18:32:37.995 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:37.994+0000 7f2f924e2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T18:32:38.039 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.037+0000 7fedd4cfa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T18:32:38.077 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.075+0000 7f2f924e2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T18:32:38.186 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.184+0000 7fedd4cfa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T18:32:38.225 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.224+0000 7f2f924e2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T18:32:38.261 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.259+0000 7fedd4cfa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T18:32:38.301 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.300+0000 7f2f924e2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T18:32:38.333 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.331+0000 7fedd4cfa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T18:32:38.373 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.372+0000 7f2f924e2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T18:32:38.407 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.405+0000 7fedd4cfa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T18:32:38.407 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:38.405+0000 7fedd4cfa180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T18:32:38.448 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.446+0000 7f2f924e2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T18:32:38.448 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-01-24T18:32:38.446+0000 7f2f924e2180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T18:32:39.430 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:39.428+0000 7fed8e1c7700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T18:32:39.433 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-01-24T18:32:39.432+0000 7fed8e1c7700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T18:32:39.840 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-01-24T18:32:39.839+0000 7fae2ac4a540 -1 osd.5 0 log_to_monitors true 2023-01-24T18:32:39.842 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-01-24T18:32:39.841+0000 7ff428614540 -1 osd.2 0 log_to_monitors true 2023-01-24T18:32:39.842 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-01-24T18:32:39.841+0000 7f94f1213540 -1 osd.1 0 log_to_monitors true 2023-01-24T18:32:39.851 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-01-24T18:32:39.849+0000 7f23ad12b540 -1 osd.7 0 log_to_monitors true 2023-01-24T18:32:39.856 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-01-24T18:32:39.855+0000 7efd94000540 -1 osd.0 0 log_to_monitors true 2023-01-24T18:32:39.857 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-01-24T18:32:39.855+0000 7fc28fb40540 -1 osd.6 0 log_to_monitors true 2023-01-24T18:32:39.861 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-01-24T18:32:39.860+0000 7f8d6d67b540 -1 osd.4 0 log_to_monitors true 2023-01-24T18:32:39.873 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-01-24T18:32:39.872+0000 7fc38b048540 -1 osd.3 0 log_to_monitors true 2023-01-24T18:32:41.433 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-01-24T18:32:41.431+0000 7f23a3d4c700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T18:32:41.433 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-01-24T18:32:41.432+0000 7f94e7e34700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T18:32:41.434 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-01-24T18:32:41.433+0000 7efd8ac21700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T18:32:41.434 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-01-24T18:32:41.433+0000 7fc381c69700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T18:32:41.435 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-01-24T18:32:41.433+0000 7ff41f235700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T18:32:41.436 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-01-24T18:32:41.435+0000 7f239eb46700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.439 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-01-24T18:32:41.438+0000 7f94e2c2e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.439 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-01-24T18:32:41.438+0000 7fc37ca63700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.442 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-01-24T18:32:41.441+0000 7efd85a1b700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.444 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-01-24T18:32:41.443+0000 7fae2306e700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T18:32:41.444 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-01-24T18:32:41.443+0000 7fc286761700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T18:32:41.445 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-01-24T18:32:41.444+0000 7ff41a02f700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.449 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-01-24T18:32:41.447+0000 7fae1c665700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:41.449 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-01-24T18:32:41.448+0000 7fc28155b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:42.306 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-01-24T18:32:42.305+0000 7f8d6429c700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T18:32:42.310 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-01-24T18:32:42.309+0000 7f8d5f096700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:32:43.774 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:32:44.087 INFO:teuthology.misc.health.smithi169.stdout: 2023-01-24T18:32:44.087 INFO:teuthology.misc.health.smithi169.stdout:{"epoch":14,"fsid":"51ff070a-6967-4dc4-8c3c-d7750347282f","created":"2023-01-24T18:32:33.111789+0000","modified":"2023-01-24T18:32:43.439819+0000","last_up_change":"2023-01-24T18:32:42.438348+0000","last_in_change":"2023-01-24T18:32:35.688856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:32:42.465696+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"83346d42-55a8-4fca-aad6-fffb1fb67752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":82430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":82430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":82430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":82430}]},"public_addr":"172.21.15.169:6804/82430","cluster_addr":"172.21.15.169:6806/82430","heartbeat_back_addr":"172.21.15.169:6810/82430","heartbeat_front_addr":"172.21.15.169:6808/82430","state":["exists","up"]},{"osd":1,"uuid":"dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":82429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6801","nonce":82429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6803","nonce":82429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":82429}]},"public_addr":"172.21.15.169:6800/82429","cluster_addr":"172.21.15.169:6801/82429","heartbeat_back_addr":"172.21.15.169:6803/82429","heartbeat_front_addr":"172.21.15.169:6802/82429","state":["exists","up"]},{"osd":2,"uuid":"4ec36bfe-4e7a-493b-908a-afa16532e172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6805","nonce":82431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":82431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":82431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":82431}]},"public_addr":"172.21.15.169:6805/82431","cluster_addr":"172.21.15.169:6807/82431","heartbeat_back_addr":"172.21.15.169:6811/82431","heartbeat_front_addr":"172.21.15.169:6809/82431","state":["exists","up"]},{"osd":3,"uuid":"3c96656a-f484-4701-85d7-eb9fe4bd57a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":82432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":82432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":82432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":82432}]},"public_addr":"172.21.15.169:6812/82432","cluster_addr":"172.21.15.169:6813/82432","heartbeat_back_addr":"172.21.15.169:6815/82432","heartbeat_front_addr":"172.21.15.169:6814/82432","state":["exists","up"]},{"osd":4,"uuid":"d96403b8-122b-443c-8ef0-2806cc1a630d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":82407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6809","nonce":82407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":82407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":82407}]},"public_addr":"172.21.15.186:6808/82407","cluster_addr":"172.21.15.186:6809/82407","heartbeat_back_addr":"172.21.15.186:6811/82407","heartbeat_front_addr":"172.21.15.186:6810/82407","state":["exists","up"]},{"osd":5,"uuid":"cf3e568a-94cc-46b3-8e48-f44cc56ab781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":82397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":82397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6803","nonce":82397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":82397}]},"public_addr":"172.21.15.186:6800/82397","cluster_addr":"172.21.15.186:6801/82397","heartbeat_back_addr":"172.21.15.186:6803/82397","heartbeat_front_addr":"172.21.15.186:6802/82397","state":["exists","up"]},{"osd":6,"uuid":"775579af-2d90-46b3-8e7a-0c1d3aaf1e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":82398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":82398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6807","nonce":82398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":82398}]},"public_addr":"172.21.15.186:6804/82398","cluster_addr":"172.21.15.186:6805/82398","heartbeat_back_addr":"172.21.15.186:6807/82398","heartbeat_front_addr":"172.21.15.186:6806/82398","state":["exists","up"]},{"osd":7,"uuid":"e1d5397a-666f-4a76-b897-91e859a2c942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":82428}]},"public_addr":"172.21.15.186:6812/82428","cluster_addr":"172.21.15.186:6813/82428","heartbeat_back_addr":"172.21.15.186:6815/82428","heartbeat_front_addr":"172.21.15.186:6814/82428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.835664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.817363+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.803275+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.886279+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.840801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.802426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.843204+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.831029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:32:44.100 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T18:32:44.100 INFO:tasks.ceph:Creating RBD pool 2023-01-24T18:32:44.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T18:32:44.450 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'rbd' created 2023-01-24T18:32:44.461 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T18:32:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T18:32:45.465 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T18:32:45.465 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T18:32:45.466 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T18:32:45.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T18:32:45.467 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T18:32:45.830 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:45.842 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":4,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":82186}]},"active_addr":"172.21.15.169:6816/82186","active_change":"2023-01-24T18:32:38.419639+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":4111701615}]},{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":3779371959}]},{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":2357945837}]}]}} 2023-01-24T18:32:45.846 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T18:32:45.846 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T18:32:45.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:32:46.149 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:46.149 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":16,"fsid":"51ff070a-6967-4dc4-8c3c-d7750347282f","created":"2023-01-24T18:32:33.111789+0000","modified":"2023-01-24T18:32:45.442446+0000","last_up_change":"2023-01-24T18:32:42.438348+0000","last_in_change":"2023-01-24T18:32:35.688856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:32:42.465696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T18:32:44.419142+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"83346d42-55a8-4fca-aad6-fffb1fb67752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":82430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":82430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":82430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":82430}]},"public_addr":"172.21.15.169:6804/82430","cluster_addr":"172.21.15.169:6806/82430","heartbeat_back_addr":"172.21.15.169:6810/82430","heartbeat_front_addr":"172.21.15.169:6808/82430","state":["exists","up"]},{"osd":1,"uuid":"dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":82429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6801","nonce":82429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6803","nonce":82429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":82429}]},"public_addr":"172.21.15.169:6800/82429","cluster_addr":"172.21.15.169:6801/82429","heartbeat_back_addr":"172.21.15.169:6803/82429","heartbeat_front_addr":"172.21.15.169:6802/82429","state":["exists","up"]},{"osd":2,"uuid":"4ec36bfe-4e7a-493b-908a-afa16532e172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6805","nonce":82431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":82431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":82431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":82431}]},"public_addr":"172.21.15.169:6805/82431","cluster_addr":"172.21.15.169:6807/82431","heartbeat_back_addr":"172.21.15.169:6811/82431","heartbeat_front_addr":"172.21.15.169:6809/82431","state":["exists","up"]},{"osd":3,"uuid":"3c96656a-f484-4701-85d7-eb9fe4bd57a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":82432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":82432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":82432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":82432}]},"public_addr":"172.21.15.169:6812/82432","cluster_addr":"172.21.15.169:6813/82432","heartbeat_back_addr":"172.21.15.169:6815/82432","heartbeat_front_addr":"172.21.15.169:6814/82432","state":["exists","up"]},{"osd":4,"uuid":"d96403b8-122b-443c-8ef0-2806cc1a630d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":82407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6809","nonce":82407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":82407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":82407}]},"public_addr":"172.21.15.186:6808/82407","cluster_addr":"172.21.15.186:6809/82407","heartbeat_back_addr":"172.21.15.186:6811/82407","heartbeat_front_addr":"172.21.15.186:6810/82407","state":["exists","up"]},{"osd":5,"uuid":"cf3e568a-94cc-46b3-8e48-f44cc56ab781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":82397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":82397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6803","nonce":82397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":82397}]},"public_addr":"172.21.15.186:6800/82397","cluster_addr":"172.21.15.186:6801/82397","heartbeat_back_addr":"172.21.15.186:6803/82397","heartbeat_front_addr":"172.21.15.186:6802/82397","state":["exists","up"]},{"osd":6,"uuid":"775579af-2d90-46b3-8e7a-0c1d3aaf1e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":82398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":82398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6807","nonce":82398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":82398}]},"public_addr":"172.21.15.186:6804/82398","cluster_addr":"172.21.15.186:6805/82398","heartbeat_back_addr":"172.21.15.186:6807/82398","heartbeat_front_addr":"172.21.15.186:6806/82398","state":["exists","up"]},{"osd":7,"uuid":"e1d5397a-666f-4a76-b897-91e859a2c942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":82428}]},"public_addr":"172.21.15.186:6812/82428","cluster_addr":"172.21.15.186:6813/82428","heartbeat_back_addr":"172.21.15.186:6815/82428","heartbeat_front_addr":"172.21.15.186:6814/82428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.835664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.817363+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.803275+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.886279+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.840801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.802426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.843204+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.831029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:32:46.162 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T18:32:46.163 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:32:46.476 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:46.477 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":17,"fsid":"51ff070a-6967-4dc4-8c3c-d7750347282f","created":"2023-01-24T18:32:33.111789+0000","modified":"2023-01-24T18:32:46.444061+0000","last_up_change":"2023-01-24T18:32:42.438348+0000","last_in_change":"2023-01-24T18:32:35.688856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T18:32:42.465696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T18:32:44.419142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"83346d42-55a8-4fca-aad6-fffb1fb67752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":82430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":82430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":82430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":82430}]},"public_addr":"172.21.15.169:6804/82430","cluster_addr":"172.21.15.169:6806/82430","heartbeat_back_addr":"172.21.15.169:6810/82430","heartbeat_front_addr":"172.21.15.169:6808/82430","state":["exists","up"]},{"osd":1,"uuid":"dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":82429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6801","nonce":82429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6803","nonce":82429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":82429}]},"public_addr":"172.21.15.169:6800/82429","cluster_addr":"172.21.15.169:6801/82429","heartbeat_back_addr":"172.21.15.169:6803/82429","heartbeat_front_addr":"172.21.15.169:6802/82429","state":["exists","up"]},{"osd":2,"uuid":"4ec36bfe-4e7a-493b-908a-afa16532e172","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6805","nonce":82431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6807","nonce":82431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":82431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":82431}]},"public_addr":"172.21.15.169:6805/82431","cluster_addr":"172.21.15.169:6807/82431","heartbeat_back_addr":"172.21.15.169:6811/82431","heartbeat_front_addr":"172.21.15.169:6809/82431","state":["exists","up"]},{"osd":3,"uuid":"3c96656a-f484-4701-85d7-eb9fe4bd57a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":82432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":82432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":82432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":82432}]},"public_addr":"172.21.15.169:6812/82432","cluster_addr":"172.21.15.169:6813/82432","heartbeat_back_addr":"172.21.15.169:6815/82432","heartbeat_front_addr":"172.21.15.169:6814/82432","state":["exists","up"]},{"osd":4,"uuid":"d96403b8-122b-443c-8ef0-2806cc1a630d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":82407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6809","nonce":82407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":82407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":82407}]},"public_addr":"172.21.15.186:6808/82407","cluster_addr":"172.21.15.186:6809/82407","heartbeat_back_addr":"172.21.15.186:6811/82407","heartbeat_front_addr":"172.21.15.186:6810/82407","state":["exists","up"]},{"osd":5,"uuid":"cf3e568a-94cc-46b3-8e48-f44cc56ab781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":82397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":82397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6803","nonce":82397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":82397}]},"public_addr":"172.21.15.186:6800/82397","cluster_addr":"172.21.15.186:6801/82397","heartbeat_back_addr":"172.21.15.186:6803/82397","heartbeat_front_addr":"172.21.15.186:6802/82397","state":["exists","up"]},{"osd":6,"uuid":"775579af-2d90-46b3-8e7a-0c1d3aaf1e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":82398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":82398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6807","nonce":82398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":82398}]},"public_addr":"172.21.15.186:6804/82398","cluster_addr":"172.21.15.186:6805/82398","heartbeat_back_addr":"172.21.15.186:6807/82398","heartbeat_front_addr":"172.21.15.186:6806/82398","state":["exists","up"]},{"osd":7,"uuid":"e1d5397a-666f-4a76-b897-91e859a2c942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":82428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":82428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":82428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":82428}]},"public_addr":"172.21.15.186:6812/82428","cluster_addr":"172.21.15.186:6813/82428","heartbeat_back_addr":"172.21.15.186:6815/82428","heartbeat_front_addr":"172.21.15.186:6814/82428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.835664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.817363+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.803275+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.886279+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.840801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.802426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.843204+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:32:40.831029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:32:46.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T18:32:46.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T18:32:46.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T18:32:46.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T18:32:46.493 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T18:32:46.493 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T18:32:46.494 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T18:32:46.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T18:32:46.764 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:46.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T18:32:46.765 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:46.765 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T18:32:46.766 INFO:teuthology.orchestra.run.smithi169.stdout:55834574852 2023-01-24T18:32:46.767 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T18:32:46.773 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:46.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T18:32:46.776 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:46.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T18:32:46.782 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:46.782 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T18:32:46.786 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:46.786 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T18:32:46.793 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:46.794 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:32:47.327 INFO:teuthology.orchestra.run.smithi169.stdout:55834574849 2023-01-24T18:32:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:47.390 INFO:teuthology.orchestra.run.smithi169.stdout:55834574849 2023-01-24T18:32:47.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2023-01-24T18:32:47.410 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:47.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.3 2023-01-24T18:32:47.412 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:47.412 INFO:teuthology.orchestra.run.smithi169.stdout:55834574849 2023-01-24T18:32:47.424 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:47.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.5 2023-01-24T18:32:47.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-24T18:32:47.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-01-24T18:32:47.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.6 2023-01-24T18:32:47.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-24T18:32:47.480 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:47.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-24T18:32:48.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T18:32:48.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T18:32:48.432 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T18:32:48.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T18:32:48.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T18:32:48.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:32:48.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T18:32:48.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T18:32:48.965 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:48.977 INFO:teuthology.orchestra.run.smithi169.stdout:55834574852 2023-01-24T18:32:49.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2023-01-24T18:32:49.018 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.3 2023-01-24T18:32:49.019 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.036 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:49.043 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:49.045 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:49.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2023-01-24T18:32:49.060 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.6 2023-01-24T18:32:49.062 INFO:teuthology.orchestra.run.smithi169.stdout:55834574850 2023-01-24T18:32:49.062 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-24T18:32:49.069 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.069 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:49.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.5 2023-01-24T18:32:49.077 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.080 INFO:teuthology.orchestra.run.smithi169.stdout:55834574851 2023-01-24T18:32:49.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-24T18:32:49.085 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2023-01-24T18:32:49.095 DEBUG:teuthology.parallel:result is None 2023-01-24T18:32:49.095 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:32:49.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:32:49.439 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:49.441 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-01-24T18:32:49.452 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T18:32:48.425240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40696,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":39488,"kb_avail":754934024,"statfs":{"total":773094113280,"available":773052440576,"internally_reserved":0,"allocated":1163264,"data_stored":875970,"data_compressed":3008,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":40435712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.454542+0000","last_change":"2023-01-24T18:32:45.454542+0000","last_active":"2023-01-24T18:32:45.454542+0000","last_peered":"2023-01-24T18:32:45.454542+0000","last_clean":"2023-01-24T18:32:45.454542+0000","last_became_active":"2023-01-24T18:32:45.454132+0000","last_became_peered":"2023-01-24T18:32:45.454132+0000","last_unstale":"2023-01-24T18:32:45.454542+0000","last_undegraded":"2023-01-24T18:32:45.454542+0000","last_fullsized":"2023-01-24T18:32:45.454542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:11.106709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.454129+0000","last_change":"2023-01-24T18:32:45.454129+0000","last_active":"2023-01-24T18:32:45.454129+0000","last_peered":"2023-01-24T18:32:45.454129+0000","last_clean":"2023-01-24T18:32:45.454129+0000","last_became_active":"2023-01-24T18:32:45.453815+0000","last_became_peered":"2023-01-24T18:32:45.453815+0000","last_unstale":"2023-01-24T18:32:45.454129+0000","last_undegraded":"2023-01-24T18:32:45.454129+0000","last_fullsized":"2023-01-24T18:32:45.454129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:57.221830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.610534+0000","last_change":"2023-01-24T18:32:45.608784+0000","last_active":"2023-01-24T18:32:45.610534+0000","last_peered":"2023-01-24T18:32:45.610534+0000","last_clean":"2023-01-24T18:32:45.610534+0000","last_became_active":"2023-01-24T18:32:45.608255+0000","last_became_peered":"2023-01-24T18:32:45.608255+0000","last_unstale":"2023-01-24T18:32:45.610534+0000","last_undegraded":"2023-01-24T18:32:45.610534+0000","last_fullsized":"2023-01-24T18:32:45.610534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.198811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.611336+0000","last_change":"2023-01-24T18:32:45.609689+0000","last_active":"2023-01-24T18:32:45.611336+0000","last_peered":"2023-01-24T18:32:45.611336+0000","last_clean":"2023-01-24T18:32:45.611336+0000","last_became_active":"2023-01-24T18:32:45.609406+0000","last_became_peered":"2023-01-24T18:32:45.609406+0000","last_unstale":"2023-01-24T18:32:45.611336+0000","last_undegraded":"2023-01-24T18:32:45.611336+0000","last_fullsized":"2023-01-24T18:32:45.611336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:00.076348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'44","reported_seq":72,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:46.420573+0000","last_change":"2023-01-24T18:32:44.457064+0000","last_active":"2023-01-24T18:32:46.420573+0000","last_peered":"2023-01-24T18:32:46.420573+0000","last_clean":"2023-01-24T18:32:46.420573+0000","last_became_active":"2023-01-24T18:32:44.456140+0000","last_became_peered":"2023-01-24T18:32:44.456140+0000","last_unstale":"2023-01-24T18:32:46.420573+0000","last_undegraded":"2023-01-24T18:32:46.420573+0000","last_fullsized":"2023-01-24T18:32:46.420573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_clean_scrub_stamp":"2023-01-24T18:32:43.439819+0000","objects_scrubbed":0,"log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.083902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455993+0000","last_change":"2023-01-24T18:32:45.455165+0000","last_active":"2023-01-24T18:32:45.455993+0000","last_peered":"2023-01-24T18:32:45.455993+0000","last_clean":"2023-01-24T18:32:45.455993+0000","last_became_active":"2023-01-24T18:32:45.454739+0000","last_became_peered":"2023-01-24T18:32:45.454739+0000","last_unstale":"2023-01-24T18:32:45.455993+0000","last_undegraded":"2023-01-24T18:32:45.455993+0000","last_fullsized":"2023-01-24T18:32:45.455993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.912370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.608478+0000","last_change":"2023-01-24T18:32:45.608478+0000","last_active":"2023-01-24T18:32:45.608478+0000","last_peered":"2023-01-24T18:32:45.608478+0000","last_clean":"2023-01-24T18:32:45.608478+0000","last_became_active":"2023-01-24T18:32:45.608228+0000","last_became_peered":"2023-01-24T18:32:45.608228+0000","last_unstale":"2023-01-24T18:32:45.608478+0000","last_undegraded":"2023-01-24T18:32:45.608478+0000","last_fullsized":"2023-01-24T18:32:45.608478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.147248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455718+0000","last_change":"2023-01-24T18:32:45.454507+0000","last_active":"2023-01-24T18:32:45.455718+0000","last_peered":"2023-01-24T18:32:45.455718+0000","last_clean":"2023-01-24T18:32:45.455718+0000","last_became_active":"2023-01-24T18:32:45.454117+0000","last_became_peered":"2023-01-24T18:32:45.454117+0000","last_unstale":"2023-01-24T18:32:45.455718+0000","last_undegraded":"2023-01-24T18:32:45.455718+0000","last_fullsized":"2023-01-24T18:32:45.455718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.826991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.608502+0000","last_change":"2023-01-24T18:32:45.608502+0000","last_active":"2023-01-24T18:32:45.608502+0000","last_peered":"2023-01-24T18:32:45.608502+0000","last_clean":"2023-01-24T18:32:45.608502+0000","last_became_active":"2023-01-24T18:32:45.608211+0000","last_became_peered":"2023-01-24T18:32:45.608211+0000","last_unstale":"2023-01-24T18:32:45.608502+0000","last_undegraded":"2023-01-24T18:32:45.608502+0000","last_fullsized":"2023-01-24T18:32:45.608502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:52.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":615984,"data_compressed":3372,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5156,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366684,"statfs":{"total":96636764160,"available":96631484416,"internally_reserved":0,"allocated":159744,"data_stored":180642,"data_compressed":760,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.5869999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.021000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":159744,"data_stored":245683,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":155648,"data_stored":244727,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:32:49.453 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:32:49.754 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:49.756 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-01-24T18:32:49.767 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T18:32:48.425240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40696,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":39488,"kb_avail":754934024,"statfs":{"total":773094113280,"available":773052440576,"internally_reserved":0,"allocated":1163264,"data_stored":875970,"data_compressed":3008,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":40435712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.454542+0000","last_change":"2023-01-24T18:32:45.454542+0000","last_active":"2023-01-24T18:32:45.454542+0000","last_peered":"2023-01-24T18:32:45.454542+0000","last_clean":"2023-01-24T18:32:45.454542+0000","last_became_active":"2023-01-24T18:32:45.454132+0000","last_became_peered":"2023-01-24T18:32:45.454132+0000","last_unstale":"2023-01-24T18:32:45.454542+0000","last_undegraded":"2023-01-24T18:32:45.454542+0000","last_fullsized":"2023-01-24T18:32:45.454542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:11.106709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.454129+0000","last_change":"2023-01-24T18:32:45.454129+0000","last_active":"2023-01-24T18:32:45.454129+0000","last_peered":"2023-01-24T18:32:45.454129+0000","last_clean":"2023-01-24T18:32:45.454129+0000","last_became_active":"2023-01-24T18:32:45.453815+0000","last_became_peered":"2023-01-24T18:32:45.453815+0000","last_unstale":"2023-01-24T18:32:45.454129+0000","last_undegraded":"2023-01-24T18:32:45.454129+0000","last_fullsized":"2023-01-24T18:32:45.454129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:57.221830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.610534+0000","last_change":"2023-01-24T18:32:45.608784+0000","last_active":"2023-01-24T18:32:45.610534+0000","last_peered":"2023-01-24T18:32:45.610534+0000","last_clean":"2023-01-24T18:32:45.610534+0000","last_became_active":"2023-01-24T18:32:45.608255+0000","last_became_peered":"2023-01-24T18:32:45.608255+0000","last_unstale":"2023-01-24T18:32:45.610534+0000","last_undegraded":"2023-01-24T18:32:45.610534+0000","last_fullsized":"2023-01-24T18:32:45.610534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.198811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.611336+0000","last_change":"2023-01-24T18:32:45.609689+0000","last_active":"2023-01-24T18:32:45.611336+0000","last_peered":"2023-01-24T18:32:45.611336+0000","last_clean":"2023-01-24T18:32:45.611336+0000","last_became_active":"2023-01-24T18:32:45.609406+0000","last_became_peered":"2023-01-24T18:32:45.609406+0000","last_unstale":"2023-01-24T18:32:45.611336+0000","last_undegraded":"2023-01-24T18:32:45.611336+0000","last_fullsized":"2023-01-24T18:32:45.611336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:00.076348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'44","reported_seq":72,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:46.420573+0000","last_change":"2023-01-24T18:32:44.457064+0000","last_active":"2023-01-24T18:32:46.420573+0000","last_peered":"2023-01-24T18:32:46.420573+0000","last_clean":"2023-01-24T18:32:46.420573+0000","last_became_active":"2023-01-24T18:32:44.456140+0000","last_became_peered":"2023-01-24T18:32:44.456140+0000","last_unstale":"2023-01-24T18:32:46.420573+0000","last_undegraded":"2023-01-24T18:32:46.420573+0000","last_fullsized":"2023-01-24T18:32:46.420573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_clean_scrub_stamp":"2023-01-24T18:32:43.439819+0000","objects_scrubbed":0,"log_size":44,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.083902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455993+0000","last_change":"2023-01-24T18:32:45.455165+0000","last_active":"2023-01-24T18:32:45.455993+0000","last_peered":"2023-01-24T18:32:45.455993+0000","last_clean":"2023-01-24T18:32:45.455993+0000","last_became_active":"2023-01-24T18:32:45.454739+0000","last_became_peered":"2023-01-24T18:32:45.454739+0000","last_unstale":"2023-01-24T18:32:45.455993+0000","last_undegraded":"2023-01-24T18:32:45.455993+0000","last_fullsized":"2023-01-24T18:32:45.455993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.912370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.608478+0000","last_change":"2023-01-24T18:32:45.608478+0000","last_active":"2023-01-24T18:32:45.608478+0000","last_peered":"2023-01-24T18:32:45.608478+0000","last_clean":"2023-01-24T18:32:45.608478+0000","last_became_active":"2023-01-24T18:32:45.608228+0000","last_became_peered":"2023-01-24T18:32:45.608228+0000","last_unstale":"2023-01-24T18:32:45.608478+0000","last_undegraded":"2023-01-24T18:32:45.608478+0000","last_fullsized":"2023-01-24T18:32:45.608478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.147248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455718+0000","last_change":"2023-01-24T18:32:45.454507+0000","last_active":"2023-01-24T18:32:45.455718+0000","last_peered":"2023-01-24T18:32:45.455718+0000","last_clean":"2023-01-24T18:32:45.455718+0000","last_became_active":"2023-01-24T18:32:45.454117+0000","last_became_peered":"2023-01-24T18:32:45.454117+0000","last_unstale":"2023-01-24T18:32:45.455718+0000","last_undegraded":"2023-01-24T18:32:45.455718+0000","last_fullsized":"2023-01-24T18:32:45.455718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.826991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.608502+0000","last_change":"2023-01-24T18:32:45.608502+0000","last_active":"2023-01-24T18:32:45.608502+0000","last_peered":"2023-01-24T18:32:45.608502+0000","last_clean":"2023-01-24T18:32:45.608502+0000","last_became_active":"2023-01-24T18:32:45.608211+0000","last_became_peered":"2023-01-24T18:32:45.608211+0000","last_unstale":"2023-01-24T18:32:45.608502+0000","last_undegraded":"2023-01-24T18:32:45.608502+0000","last_fullsized":"2023-01-24T18:32:45.608502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:52.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328208,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":73,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":615984,"data_compressed":3372,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5156,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366684,"statfs":{"total":96636764160,"available":96631484416,"internally_reserved":0,"allocated":159744,"data_stored":180642,"data_compressed":760,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.5869999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.021000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5224,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366616,"statfs":{"total":96636764160,"available":96631414784,"internally_reserved":0,"allocated":159744,"data_stored":245683,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":155648,"data_stored":244727,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":205328,"data_compressed":1124,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:32:49.768 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T18:32:49.768 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T18:32:49.769 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T18:32:49.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T18:32:50.145 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:50.145 INFO:teuthology.orchestra.run.smithi169.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T18:32:50.159 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T18:32:50.159 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-24T18:32:50.170 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-24T18:32:50.170 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T18:32:50.170 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-24T18:32:50.171 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-24T18:32:50.494 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-24T18:32:50.495 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:32:50.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:32:50.795 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:50.797 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-01-24T18:32:50.808 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T18:32:50.425574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41548,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":40192,"kb_avail":754933172,"statfs":{"total":773094113280,"available":773051568128,"internally_reserved":0,"allocated":1273856,"data_stored":1627638,"data_compressed":17868,"data_compressed_allocated":77824,"data_compressed_original":1228800,"omap_allocated":0,"internal_metadata":41156608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262160,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":47,"num_write":86,"num_write_kb":1761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000333"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455359+0000","last_change":"2023-01-24T18:32:45.454542+0000","last_active":"2023-01-24T18:32:45.455359+0000","last_peered":"2023-01-24T18:32:45.455359+0000","last_clean":"2023-01-24T18:32:45.455359+0000","last_became_active":"2023-01-24T18:32:45.454132+0000","last_became_peered":"2023-01-24T18:32:45.454132+0000","last_unstale":"2023-01-24T18:32:45.455359+0000","last_undegraded":"2023-01-24T18:32:45.455359+0000","last_fullsized":"2023-01-24T18:32:45.455359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:11.106709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455366+0000","last_change":"2023-01-24T18:32:45.454129+0000","last_active":"2023-01-24T18:32:45.455366+0000","last_peered":"2023-01-24T18:32:45.455366+0000","last_clean":"2023-01-24T18:32:45.455366+0000","last_became_active":"2023-01-24T18:32:45.453815+0000","last_became_peered":"2023-01-24T18:32:45.453815+0000","last_unstale":"2023-01-24T18:32:45.455366+0000","last_undegraded":"2023-01-24T18:32:45.455366+0000","last_fullsized":"2023-01-24T18:32:45.455366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:57.221830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.610534+0000","last_change":"2023-01-24T18:32:45.608784+0000","last_active":"2023-01-24T18:32:45.610534+0000","last_peered":"2023-01-24T18:32:45.610534+0000","last_clean":"2023-01-24T18:32:45.610534+0000","last_became_active":"2023-01-24T18:32:45.608255+0000","last_became_peered":"2023-01-24T18:32:45.608255+0000","last_unstale":"2023-01-24T18:32:45.610534+0000","last_undegraded":"2023-01-24T18:32:45.610534+0000","last_fullsized":"2023-01-24T18:32:45.610534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.198811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.611336+0000","last_change":"2023-01-24T18:32:45.609689+0000","last_active":"2023-01-24T18:32:45.611336+0000","last_peered":"2023-01-24T18:32:45.611336+0000","last_clean":"2023-01-24T18:32:45.611336+0000","last_became_active":"2023-01-24T18:32:45.609406+0000","last_became_peered":"2023-01-24T18:32:45.609406+0000","last_unstale":"2023-01-24T18:32:45.611336+0000","last_undegraded":"2023-01-24T18:32:45.611336+0000","last_fullsized":"2023-01-24T18:32:45.611336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:00.076348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:32:48.234423+0000","last_change":"2023-01-24T18:32:44.457064+0000","last_active":"2023-01-24T18:32:48.234423+0000","last_peered":"2023-01-24T18:32:48.234423+0000","last_clean":"2023-01-24T18:32:48.234423+0000","last_became_active":"2023-01-24T18:32:44.456140+0000","last_became_peered":"2023-01-24T18:32:44.456140+0000","last_unstale":"2023-01-24T18:32:48.234423+0000","last_undegraded":"2023-01-24T18:32:48.234423+0000","last_fullsized":"2023-01-24T18:32:48.234423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_clean_scrub_stamp":"2023-01-24T18:32:43.439819+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.083902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455993+0000","last_change":"2023-01-24T18:32:45.455165+0000","last_active":"2023-01-24T18:32:45.455993+0000","last_peered":"2023-01-24T18:32:45.455993+0000","last_clean":"2023-01-24T18:32:45.455993+0000","last_became_active":"2023-01-24T18:32:45.454739+0000","last_became_peered":"2023-01-24T18:32:45.454739+0000","last_unstale":"2023-01-24T18:32:45.455993+0000","last_undegraded":"2023-01-24T18:32:45.455993+0000","last_fullsized":"2023-01-24T18:32:45.455993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.912370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.609658+0000","last_change":"2023-01-24T18:32:45.608478+0000","last_active":"2023-01-24T18:32:45.609658+0000","last_peered":"2023-01-24T18:32:45.609658+0000","last_clean":"2023-01-24T18:32:45.609658+0000","last_became_active":"2023-01-24T18:32:45.608228+0000","last_became_peered":"2023-01-24T18:32:45.608228+0000","last_unstale":"2023-01-24T18:32:45.609658+0000","last_undegraded":"2023-01-24T18:32:45.609658+0000","last_fullsized":"2023-01-24T18:32:45.609658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.147248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455718+0000","last_change":"2023-01-24T18:32:45.454507+0000","last_active":"2023-01-24T18:32:45.455718+0000","last_peered":"2023-01-24T18:32:45.455718+0000","last_clean":"2023-01-24T18:32:45.455718+0000","last_became_active":"2023-01-24T18:32:45.454117+0000","last_became_peered":"2023-01-24T18:32:45.454117+0000","last_unstale":"2023-01-24T18:32:45.455718+0000","last_undegraded":"2023-01-24T18:32:45.455718+0000","last_fullsized":"2023-01-24T18:32:45.455718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.826991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.609710+0000","last_change":"2023-01-24T18:32:45.608502+0000","last_active":"2023-01-24T18:32:45.609710+0000","last_peered":"2023-01-24T18:32:45.609710+0000","last_clean":"2023-01-24T18:32:45.609710+0000","last_became_active":"2023-01-24T18:32:45.608211+0000","last_became_peered":"2023-01-24T18:32:45.608211+0000","last_unstale":"2023-01-24T18:32:45.609710+0000","last_undegraded":"2023-01-24T18:32:45.609710+0000","last_fullsized":"2023-01-24T18:32:45.609710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:52.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":1586784,"data_compressed":24837,"data_compressed_allocated":98304,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":196608,"data_stored":513470,"data_compressed":7948,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.021000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5468,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366372,"statfs":{"total":96636764160,"available":96631164928,"internally_reserved":0,"allocated":188416,"data_stored":450487,"data_compressed":4960,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5012,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366828,"statfs":{"total":96636764160,"available":96631631872,"internally_reserved":0,"allocated":143360,"data_stored":45026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5372,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366468,"statfs":{"total":96636764160,"available":96631263232,"internally_reserved":0,"allocated":184320,"data_stored":449531,"data_compressed":4960,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:32:50.809 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:32:51.112 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-01-24T18:32:51.114 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-01-24T18:32:51.125 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T18:32:50.425574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41548,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":40192,"kb_avail":754933172,"statfs":{"total":773094113280,"available":773051568128,"internally_reserved":0,"allocated":1273856,"data_stored":1627638,"data_compressed":17868,"data_compressed_allocated":77824,"data_compressed_original":1228800,"omap_allocated":0,"internal_metadata":41156608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262160,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":47,"num_write":86,"num_write_kb":1761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000333"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455359+0000","last_change":"2023-01-24T18:32:45.454542+0000","last_active":"2023-01-24T18:32:45.455359+0000","last_peered":"2023-01-24T18:32:45.455359+0000","last_clean":"2023-01-24T18:32:45.455359+0000","last_became_active":"2023-01-24T18:32:45.454132+0000","last_became_peered":"2023-01-24T18:32:45.454132+0000","last_unstale":"2023-01-24T18:32:45.455359+0000","last_undegraded":"2023-01-24T18:32:45.455359+0000","last_fullsized":"2023-01-24T18:32:45.455359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:11.106709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455366+0000","last_change":"2023-01-24T18:32:45.454129+0000","last_active":"2023-01-24T18:32:45.455366+0000","last_peered":"2023-01-24T18:32:45.455366+0000","last_clean":"2023-01-24T18:32:45.455366+0000","last_became_active":"2023-01-24T18:32:45.453815+0000","last_became_peered":"2023-01-24T18:32:45.453815+0000","last_unstale":"2023-01-24T18:32:45.455366+0000","last_undegraded":"2023-01-24T18:32:45.455366+0000","last_fullsized":"2023-01-24T18:32:45.455366+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:57.221830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.610534+0000","last_change":"2023-01-24T18:32:45.608784+0000","last_active":"2023-01-24T18:32:45.610534+0000","last_peered":"2023-01-24T18:32:45.610534+0000","last_clean":"2023-01-24T18:32:45.610534+0000","last_became_active":"2023-01-24T18:32:45.608255+0000","last_became_peered":"2023-01-24T18:32:45.608255+0000","last_unstale":"2023-01-24T18:32:45.610534+0000","last_undegraded":"2023-01-24T18:32:45.610534+0000","last_fullsized":"2023-01-24T18:32:45.610534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.198811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.611336+0000","last_change":"2023-01-24T18:32:45.609689+0000","last_active":"2023-01-24T18:32:45.611336+0000","last_peered":"2023-01-24T18:32:45.611336+0000","last_clean":"2023-01-24T18:32:45.611336+0000","last_became_active":"2023-01-24T18:32:45.609406+0000","last_became_peered":"2023-01-24T18:32:45.609406+0000","last_unstale":"2023-01-24T18:32:45.611336+0000","last_undegraded":"2023-01-24T18:32:45.611336+0000","last_fullsized":"2023-01-24T18:32:45.611336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:34:00.076348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T18:32:48.234423+0000","last_change":"2023-01-24T18:32:44.457064+0000","last_active":"2023-01-24T18:32:48.234423+0000","last_peered":"2023-01-24T18:32:48.234423+0000","last_clean":"2023-01-24T18:32:48.234423+0000","last_became_active":"2023-01-24T18:32:44.456140+0000","last_became_peered":"2023-01-24T18:32:44.456140+0000","last_unstale":"2023-01-24T18:32:48.234423+0000","last_undegraded":"2023-01-24T18:32:48.234423+0000","last_fullsized":"2023-01-24T18:32:48.234423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:43.439819+0000","last_clean_scrub_stamp":"2023-01-24T18:32:43.439819+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.083902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455993+0000","last_change":"2023-01-24T18:32:45.455165+0000","last_active":"2023-01-24T18:32:45.455993+0000","last_peered":"2023-01-24T18:32:45.455993+0000","last_clean":"2023-01-24T18:32:45.455993+0000","last_became_active":"2023-01-24T18:32:45.454739+0000","last_became_peered":"2023-01-24T18:32:45.454739+0000","last_unstale":"2023-01-24T18:32:45.455993+0000","last_undegraded":"2023-01-24T18:32:45.455993+0000","last_fullsized":"2023-01-24T18:32:45.455993+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.912370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.609658+0000","last_change":"2023-01-24T18:32:45.608478+0000","last_active":"2023-01-24T18:32:45.609658+0000","last_peered":"2023-01-24T18:32:45.609658+0000","last_clean":"2023-01-24T18:32:45.609658+0000","last_became_active":"2023-01-24T18:32:45.608228+0000","last_became_peered":"2023-01-24T18:32:45.608228+0000","last_unstale":"2023-01-24T18:32:45.609658+0000","last_undegraded":"2023-01-24T18:32:45.609658+0000","last_fullsized":"2023-01-24T18:32:45.609658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:56.147248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.455718+0000","last_change":"2023-01-24T18:32:45.454507+0000","last_active":"2023-01-24T18:32:45.455718+0000","last_peered":"2023-01-24T18:32:45.455718+0000","last_clean":"2023-01-24T18:32:45.455718+0000","last_became_active":"2023-01-24T18:32:45.454117+0000","last_became_peered":"2023-01-24T18:32:45.454117+0000","last_unstale":"2023-01-24T18:32:45.455718+0000","last_undegraded":"2023-01-24T18:32:45.455718+0000","last_fullsized":"2023-01-24T18:32:45.455718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:54.826991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T18:32:45.609710+0000","last_change":"2023-01-24T18:32:45.608502+0000","last_active":"2023-01-24T18:32:45.609710+0000","last_peered":"2023-01-24T18:32:45.609710+0000","last_clean":"2023-01-24T18:32:45.609710+0000","last_became_active":"2023-01-24T18:32:45.608211+0000","last_became_peered":"2023-01-24T18:32:45.608211+0000","last_unstale":"2023-01-24T18:32:45.609710+0000","last_undegraded":"2023-01-24T18:32:45.609710+0000","last_fullsized":"2023-01-24T18:32:45.609710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:32:44.441003+0000","last_clean_scrub_stamp":"2023-01-24T18:32:44.441003+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T18:33:52.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":1586784,"data_compressed":24837,"data_compressed_allocated":98304,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":40410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":196608,"data_stored":513470,"data_compressed":7948,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.021000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5468,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366372,"statfs":{"total":96636764160,"available":96631164928,"internally_reserved":0,"allocated":188416,"data_stored":450487,"data_compressed":4960,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5012,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366828,"statfs":{"total":96636764160,"available":96631631872,"internally_reserved":0,"allocated":143360,"data_stored":45026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5372,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366468,"statfs":{"total":96636764160,"available":96631263232,"internally_reserved":0,"allocated":184320,"data_stored":449531,"data_compressed":4960,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":8279,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:32:51.126 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T18:32:51.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-24T18:32:51.436 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 17 2023-01-24T18:32:51.436 INFO:teuthology.orchestra.run.smithi169.stdout:fsid 51ff070a-6967-4dc4-8c3c-d7750347282f 2023-01-24T18:32:51.437 INFO:teuthology.orchestra.run.smithi169.stdout:created 2023-01-24T18:32:33.111789+0000 2023-01-24T18:32:51.437 INFO:teuthology.orchestra.run.smithi169.stdout:modified 2023-01-24T18:32:46.444061+0000 2023-01-24T18:32:51.437 INFO:teuthology.orchestra.run.smithi169.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-24T18:32:51.437 INFO:teuthology.orchestra.run.smithi169.stdout:crush_version 4 2023-01-24T18:32:51.437 INFO:teuthology.orchestra.run.smithi169.stdout:full_ratio 0.95 2023-01-24T18:32:51.438 INFO:teuthology.orchestra.run.smithi169.stdout:backfillfull_ratio 0.9 2023-01-24T18:32:51.438 INFO:teuthology.orchestra.run.smithi169.stdout:nearfull_ratio 0.85 2023-01-24T18:32:51.438 INFO:teuthology.orchestra.run.smithi169.stdout:require_min_compat_client luminous 2023-01-24T18:32:51.438 INFO:teuthology.orchestra.run.smithi169.stdout:min_compat_client jewel 2023-01-24T18:32:51.438 INFO:teuthology.orchestra.run.smithi169.stdout:require_osd_release quincy 2023-01-24T18:32:51.439 INFO:teuthology.orchestra.run.smithi169.stdout:stretch_mode_enabled false 2023-01-24T18:32:51.439 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-24T18:32:51.439 INFO:teuthology.orchestra.run.smithi169.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-24T18:32:51.439 INFO:teuthology.orchestra.run.smithi169.stdout:max_osd 8 2023-01-24T18:32:51.439 INFO:teuthology.orchestra.run.smithi169.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6804/82430 v2:172.21.15.169:6806/82430 exists,up 83346d42-55a8-4fca-aad6-fffb1fb67752 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6800/82429 v2:172.21.15.169:6801/82429 exists,up dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6805/82431 v2:172.21.15.169:6807/82431 exists,up 4ec36bfe-4e7a-493b-908a-afa16532e172 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6812/82432 v2:172.21.15.169:6813/82432 exists,up 3c96656a-f484-4701-85d7-eb9fe4bd57a2 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6808/82407 v2:172.21.15.186:6809/82407 exists,up d96403b8-122b-443c-8ef0-2806cc1a630d 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6800/82397 v2:172.21.15.186:6801/82397 exists,up cf3e568a-94cc-46b3-8e48-f44cc56ab781 2023-01-24T18:32:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6804/82398 v2:172.21.15.186:6805/82398 exists,up 775579af-2d90-46b3-8e7a-0c1d3aaf1e69 2023-01-24T18:32:51.441 INFO:teuthology.orchestra.run.smithi169.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6812/82428 v2:172.21.15.186:6813/82428 exists,up e1d5397a-666f-4a76-b897-91e859a2c942 2023-01-24T18:32:51.447 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6804/82430 v2:172.21.15.169:6806/82430 exists,up 83346d42-55a8-4fca-aad6-fffb1fb67752', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6800/82429 v2:172.21.15.169:6801/82429 exists,up dd56cee9-1e8b-4995-bcbf-fa0a5d3d5d36', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6805/82431 v2:172.21.15.169:6807/82431 exists,up 4ec36bfe-4e7a-493b-908a-afa16532e172', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.169:6812/82432 v2:172.21.15.169:6813/82432 exists,up 3c96656a-f484-4701-85d7-eb9fe4bd57a2', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6808/82407 v2:172.21.15.186:6809/82407 exists,up d96403b8-122b-443c-8ef0-2806cc1a630d', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6800/82397 v2:172.21.15.186:6801/82397 exists,up cf3e568a-94cc-46b3-8e48-f44cc56ab781', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6804/82398 v2:172.21.15.186:6805/82398 exists,up 775579af-2d90-46b3-8e7a-0c1d3aaf1e69', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.186:6812/82428 v2:172.21.15.186:6813/82428 exists,up e1d5397a-666f-4a76-b897-91e859a2c942'] 2023-01-24T18:32:51.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-24T18:32:51.562 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-01-24T18:32:51.562 INFO:teuthology.orchestra.run.smithi169.stdout: "name": "mon.a", 2023-01-24T18:32:51.562 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster": "ceph", 2023-01-24T18:32:51.563 INFO:teuthology.orchestra.run.smithi169.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-24T18:32:51.563 INFO:teuthology.orchestra.run.smithi169.stdout: "admin_socket_mode": "", 2023-01-24T18:32:51.563 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-24T18:32:51.563 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_client_required": "cephx, none", 2023-01-24T18:32:51.563 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_cluster_required": "cephx", 2023-01-24T18:32:51.564 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_debug": "true", 2023-01-24T18:32:51.564 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-24T18:32:51.564 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-24T18:32:51.564 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_service_required": "cephx", 2023-01-24T18:32:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-24T18:32:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_supported": "cephx", 2023-01-24T18:32:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_aio": "true", 2023-01-24T18:32:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-24T18:32:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_aio_poll_ms": "250", 2023-01-24T18:32:51.566 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_aio_reap_max": "16", 2023-01-24T18:32:51.566 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_async_discard": "false", 2023-01-24T18:32:51.566 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_block_size": "4096", 2023-01-24T18:32:51.566 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_debug_aio": "false", 2023-01-24T18:32:51.566 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-24T18:32:51.567 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-24T18:32:51.567 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_debug_inflight_ios": "false", 2023-01-24T18:32:51.567 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_enable_discard": "false", 2023-01-24T18:32:51.567 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_flock_retry": "3", 2023-01-24T18:32:51.567 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-24T18:32:51.568 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_inject_crash": "0", 2023-01-24T18:32:51.568 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-24T18:32:51.568 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_ioring": "false", 2023-01-24T18:32:51.568 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_ioring_hipri": "false", 2023-01-24T18:32:51.569 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-24T18:32:51.569 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-24T18:32:51.569 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-24T18:32:51.569 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-24T18:32:51.569 INFO:teuthology.orchestra.run.smithi169.stdout: "bdev_type": "", 2023-01-24T18:32:51.570 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_alloc_size": "1048576", 2023-01-24T18:32:51.570 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_allocator": "hybrid", 2023-01-24T18:32:51.570 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_buffered_io": "true", 2023-01-24T18:32:51.570 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_check_for_zeros": "false", 2023-01-24T18:32:51.570 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-24T18:32:51.571 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-24T18:32:51.571 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_compact_log_sync": "false", 2023-01-24T18:32:51.571 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-24T18:32:51.571 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-24T18:32:51.571 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-24T18:32:51.572 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_max_log_runway": "4194304", 2023-01-24T18:32:51.572 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_max_prefetch": "1048576", 2023-01-24T18:32:51.572 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_min_flush_size": "524288", 2023-01-24T18:32:51.572 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_min_log_runway": "1048576", 2023-01-24T18:32:51.572 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_replay_recovery": "false", 2023-01-24T18:32:51.573 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-24T18:32:51.573 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-24T18:32:51.573 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_sync_write": "false", 2023-01-24T18:32:51.573 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-24T18:32:51.573 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-24T18:32:51.574 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-24T18:32:51.574 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_allocation_from_file": "true", 2023-01-24T18:32:51.574 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_allocator": "hybrid", 2023-01-24T18:32:51.574 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-24T18:32:51.574 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-24T18:32:51.575 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-24T18:32:51.575 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-24T18:32:51.575 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-24T18:32:51.575 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-24T18:32:51.576 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-24T18:32:51.576 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_create": "true", 2023-01-24T18:32:51.576 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_db_create": "false", 2023-01-24T18:32:51.576 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_db_path": "", 2023-01-24T18:32:51.576 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_db_size": "0", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_path": "", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_preallocate_file": "false", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_size": "107374182400", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_wal_create": "false", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_wal_path": "", 2023-01-24T18:32:51.577 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_block_wal_size": "100663296", 2023-01-24T18:32:51.578 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bluefs": "true", 2023-01-24T18:32:51.578 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-24T18:32:51.578 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-24T18:32:51.578 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-24T18:32:51.578 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-24T18:32:51.579 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:32:51.579 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:32:51.579 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-24T18:32:51.579 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T18:32:51.579 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_autotune": "true", 2023-01-24T18:32:51.580 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-24T18:32:51.580 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-24T18:32:51.580 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-24T18:32:51.580 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-24T18:32:51.580 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_size": "0", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_type": "2q", 2023-01-24T18:32:51.581 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-24T18:32:51.582 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_clone_cow": "true", 2023-01-24T18:32:51.582 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-24T18:32:51.582 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-24T18:32:51.582 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-24T18:32:51.582 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-24T18:32:51.583 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-24T18:32:51.583 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-24T18:32:51.583 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-24T18:32:51.583 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_mode": "none", 2023-01-24T18:32:51.583 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-24T18:32:51.584 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_csum_type": "crc32c", 2023-01-24T18:32:51.584 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-24T18:32:51.584 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_freelist": "false", 2023-01-24T18:32:51.584 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-24T18:32:51.584 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-24T18:32:51.585 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-24T18:32:51.586 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-24T18:32:51.586 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-24T18:32:51.586 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-24T18:32:51.586 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-24T18:32:51.586 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-24T18:32:51.587 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_small_allocations": "0", 2023-01-24T18:32:51.587 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-24T18:32:51.587 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_default_buffered_read": "true", 2023-01-24T18:32:51.587 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_default_buffered_write": "false", 2023-01-24T18:32:51.587 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-24T18:32:51.588 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-24T18:32:51.588 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-24T18:32:51.588 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-24T18:32:51.588 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-24T18:32:51.588 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-24T18:32:51.589 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-24T18:32:51.589 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-24T18:32:51.589 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fail_eio": "false", 2023-01-24T18:32:51.589 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-24T18:32:51.589 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_mount": "false", 2023-01-24T18:32:51.590 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-24T18:32:51.591 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_umount": "false", 2023-01-24T18:32:51.591 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-24T18:32:51.591 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-24T18:32:51.591 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-24T18:32:51.591 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-24T18:32:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-24T18:32:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-24T18:32:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-24T18:32:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-24T18:32:51.592 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_ignore_data_csum": "false", 2023-01-24T18:32:51.593 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-24T18:32:51.593 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-24T18:32:51.593 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-24T18:32:51.593 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-24T18:32:51.593 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_log_op_age": "5.000000", 2023-01-24T18:32:51.594 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_alloc_size": "0", 2023-01-24T18:32:51.594 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_blob_size": "0", 2023-01-24T18:32:51.594 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-24T18:32:51.594 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-24T18:32:51.594 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_max_deferred_txc": "32", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_min_alloc_size": "0", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_nid_prealloc": "1024", 2023-01-24T18:32:51.595 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-24T18:32:51.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-24T18:32:51.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-24T18:32:51.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-24T18:32:51.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_retry_disk_reads": "3", 2023-01-24T18:32:51.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_rocksdb_cf": "true", 2023-01-24T18:32:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-24T18:32:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2023-01-24T18:32:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-24T18:32:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-24T18:32:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-24T18:32:51.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-24T18:32:51.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_spdk_mem": "512", 2023-01-24T18:32:51.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-24T18:32:51.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-24T18:32:51.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_tracing": "false", 2023-01-24T18:32:51.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-24T18:32:51.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-24T18:32:51.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-24T18:32:51.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-24T18:32:51.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-24T18:32:51.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-24T18:32:51.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-24T18:32:51.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-24T18:32:51.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-24T18:32:51.601 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_zero_block_detection": "false", 2023-01-24T18:32:51.601 INFO:teuthology.orchestra.run.smithi169.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-24T18:32:51.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-24T18:32:51.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-24T18:32:51.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-24T18:32:51.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-24T18:32:51.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-24T18:32:51.603 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-24T18:32:51.603 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-24T18:32:51.603 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-24T18:32:51.603 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-24T18:32:51.603 INFO:teuthology.orchestra.run.smithi169.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-24T18:32:51.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-24T18:32:51.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-24T18:32:51.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_cluster_require_signatures": "false", 2023-01-24T18:32:51.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_cluster_require_version": "2", 2023-01-24T18:32:51.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_require_signatures": "false", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_require_version": "2", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_service_require_signatures": "false", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_service_require_version": "2", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cephx_sign_messages": "true", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "chdir": "", 2023-01-24T18:32:51.605 INFO:teuthology.orchestra.run.smithi169.stdout: "client_acl_type": "", 2023-01-24T18:32:51.606 INFO:teuthology.orchestra.run.smithi169.stdout: "client_asio_thread_count": "2", 2023-01-24T18:32:51.606 INFO:teuthology.orchestra.run.smithi169.stdout: "client_cache_mid": "0.750000", 2023-01-24T18:32:51.606 INFO:teuthology.orchestra.run.smithi169.stdout: "client_cache_size": "16384", 2023-01-24T18:32:51.606 INFO:teuthology.orchestra.run.smithi169.stdout: "client_caps_release_delay": "5", 2023-01-24T18:32:51.606 INFO:teuthology.orchestra.run.smithi169.stdout: "client_check_pool_perm": "true", 2023-01-24T18:32:51.607 INFO:teuthology.orchestra.run.smithi169.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-24T18:32:51.607 INFO:teuthology.orchestra.run.smithi169.stdout: "client_debug_force_sync_read": "false", 2023-01-24T18:32:51.607 INFO:teuthology.orchestra.run.smithi169.stdout: "client_debug_getattr_caps": "false", 2023-01-24T18:32:51.607 INFO:teuthology.orchestra.run.smithi169.stdout: "client_debug_inject_tick_delay": "0", 2023-01-24T18:32:51.607 INFO:teuthology.orchestra.run.smithi169.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-24T18:32:51.608 INFO:teuthology.orchestra.run.smithi169.stdout: "client_die_on_failed_remount": "false", 2023-01-24T18:32:51.608 INFO:teuthology.orchestra.run.smithi169.stdout: "client_dirsize_rbytes": "true", 2023-01-24T18:32:51.608 INFO:teuthology.orchestra.run.smithi169.stdout: "client_force_lazyio": "false", 2023-01-24T18:32:51.608 INFO:teuthology.orchestra.run.smithi169.stdout: "client_fs": "", 2023-01-24T18:32:51.608 INFO:teuthology.orchestra.run.smithi169.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-24T18:32:51.609 INFO:teuthology.orchestra.run.smithi169.stdout: "client_inject_release_failure": "false", 2023-01-24T18:32:51.609 INFO:teuthology.orchestra.run.smithi169.stdout: "client_max_inline_size": "4096", 2023-01-24T18:32:51.609 INFO:teuthology.orchestra.run.smithi169.stdout: "client_mds_namespace": "", 2023-01-24T18:32:51.609 INFO:teuthology.orchestra.run.smithi169.stdout: "client_metadata": "", 2023-01-24T18:32:51.609 INFO:teuthology.orchestra.run.smithi169.stdout: "client_mount_gid": "-1", 2023-01-24T18:32:51.610 INFO:teuthology.orchestra.run.smithi169.stdout: "client_mount_timeout": "300", 2023-01-24T18:32:51.610 INFO:teuthology.orchestra.run.smithi169.stdout: "client_mount_uid": "-1", 2023-01-24T18:32:51.610 INFO:teuthology.orchestra.run.smithi169.stdout: "client_mountpoint": "/", 2023-01-24T18:32:51.610 INFO:teuthology.orchestra.run.smithi169.stdout: "client_notify_timeout": "10", 2023-01-24T18:32:51.610 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc": "true", 2023-01-24T18:32:51.611 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc_max_dirty": "104857600", 2023-01-24T18:32:51.611 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-24T18:32:51.611 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc_max_objects": "1000", 2023-01-24T18:32:51.611 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc_size": "209715200", 2023-01-24T18:32:51.611 INFO:teuthology.orchestra.run.smithi169.stdout: "client_oc_target_dirty": "8388608", 2023-01-24T18:32:51.612 INFO:teuthology.orchestra.run.smithi169.stdout: "client_permissions": "true", 2023-01-24T18:32:51.612 INFO:teuthology.orchestra.run.smithi169.stdout: "client_quota": "true", 2023-01-24T18:32:51.612 INFO:teuthology.orchestra.run.smithi169.stdout: "client_quota_df": "true", 2023-01-24T18:32:51.612 INFO:teuthology.orchestra.run.smithi169.stdout: "client_readahead_max_bytes": "0", 2023-01-24T18:32:51.612 INFO:teuthology.orchestra.run.smithi169.stdout: "client_readahead_max_periods": "4", 2023-01-24T18:32:51.613 INFO:teuthology.orchestra.run.smithi169.stdout: "client_readahead_min": "131072", 2023-01-24T18:32:51.613 INFO:teuthology.orchestra.run.smithi169.stdout: "client_reconnect_stale": "false", 2023-01-24T18:32:51.613 INFO:teuthology.orchestra.run.smithi169.stdout: "client_shutdown_timeout": "30", 2023-01-24T18:32:51.613 INFO:teuthology.orchestra.run.smithi169.stdout: "client_snapdir": ".snap", 2023-01-24T18:32:51.613 INFO:teuthology.orchestra.run.smithi169.stdout: "client_tick_interval": "1", 2023-01-24T18:32:51.614 INFO:teuthology.orchestra.run.smithi169.stdout: "client_trace": "", 2023-01-24T18:32:51.614 INFO:teuthology.orchestra.run.smithi169.stdout: "client_try_dentry_invalidate": "false", 2023-01-24T18:32:51.614 INFO:teuthology.orchestra.run.smithi169.stdout: "client_use_faked_inos": "false", 2023-01-24T18:32:51.614 INFO:teuthology.orchestra.run.smithi169.stdout: "client_use_random_mds": "false", 2023-01-24T18:32:51.614 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_graylog": "false", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_graylog_port": "12201", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_monitors": "default=true", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_syslog": "false", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-24T18:32:51.615 INFO:teuthology.orchestra.run.smithi169.stdout: "clog_to_syslog_level": "info", 2023-01-24T18:32:51.616 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "-", 2023-01-24T18:32:51.616 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_network": "", 2023-01-24T18:32:51.616 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_network_interface": "", 2023-01-24T18:32:51.616 INFO:teuthology.orchestra.run.smithi169.stdout: "compressor_zlib_isal": "false", 2023-01-24T18:32:51.616 INFO:teuthology.orchestra.run.smithi169.stdout: "compressor_zlib_level": "5", 2023-01-24T18:32:51.617 INFO:teuthology.orchestra.run.smithi169.stdout: "compressor_zlib_winsize": "-15", 2023-01-24T18:32:51.617 INFO:teuthology.orchestra.run.smithi169.stdout: "compressor_zstd_level": "1", 2023-01-24T18:32:51.617 INFO:teuthology.orchestra.run.smithi169.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-24T18:32:51.617 INFO:teuthology.orchestra.run.smithi169.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-24T18:32:51.617 INFO:teuthology.orchestra.run.smithi169.stdout: "crimson_alien_op_num_threads": "6", 2023-01-24T18:32:51.618 INFO:teuthology.orchestra.run.smithi169.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-24T18:32:51.618 INFO:teuthology.orchestra.run.smithi169.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-24T18:32:51.618 INFO:teuthology.orchestra.run.smithi169.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-24T18:32:51.618 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_location": "", 2023-01-24T18:32:51.618 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_location_hook": "", 2023-01-24T18:32:51.619 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_location_hook_timeout": "10", 2023-01-24T18:32:51.619 INFO:teuthology.orchestra.run.smithi169.stdout: "daemonize": "false", 2023-01-24T18:32:51.619 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_alienstore": "0/5", 2023-01-24T18:32:51.619 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_allow_any_pool_priority": "false", 2023-01-24T18:32:51.619 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_asok": "1/5", 2023-01-24T18:32:51.620 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_asok_assert_abort": "false", 2023-01-24T18:32:51.620 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_asserts_on_shutdown": "true", 2023-01-24T18:32:51.620 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_auth": "20/20", 2023-01-24T18:32:51.620 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_bdev": "1/3", 2023-01-24T18:32:51.620 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_bluefs": "1/5", 2023-01-24T18:32:51.621 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_bluestore": "1/5", 2023-01-24T18:32:51.621 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_buffer": "0/1", 2023-01-24T18:32:51.621 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_ceph_exporter": "1/5", 2023-01-24T18:32:51.621 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_cephfs_mirror": "0/5", 2023-01-24T18:32:51.621 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_cephsqlite": "0/5", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_civetweb": "1/10", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_client": "0/5", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_compressor": "1/5", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_context": "0/1", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_crush": "1/1", 2023-01-24T18:32:51.622 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_crypto": "1/5", 2023-01-24T18:32:51.623 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_deliberately_leak_memory": "false", 2023-01-24T18:32:51.623 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_disable_randomized_ping": "false", 2023-01-24T18:32:51.623 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_dpdk": "1/5", 2023-01-24T18:32:51.623 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_eventtrace": "1/5", 2023-01-24T18:32:51.623 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_filer": "0/1", 2023-01-24T18:32:51.624 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_filestore": "1/3", 2023-01-24T18:32:51.624 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_finisher": "1/1", 2023-01-24T18:32:51.624 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_fuse": "1/5", 2023-01-24T18:32:51.624 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_heartbeat_testing_span": "0", 2023-01-24T18:32:51.624 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_heartbeatmap": "1/5", 2023-01-24T18:32:51.625 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-24T18:32:51.625 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_javaclient": "1/5", 2023-01-24T18:32:51.625 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_journal": "1/3", 2023-01-24T18:32:51.625 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_journaler": "0/5", 2023-01-24T18:32:51.625 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_kstore": "1/5", 2023-01-24T18:32:51.626 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_leveldb": "4/5", 2023-01-24T18:32:51.626 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_lockdep": "0/1", 2023-01-24T18:32:51.626 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mclock": "1/5", 2023-01-24T18:32:51.626 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds": "1/5", 2023-01-24T18:32:51.626 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds_balancer": "1/5", 2023-01-24T18:32:51.627 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds_locker": "1/5", 2023-01-24T18:32:51.627 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds_log": "1/5", 2023-01-24T18:32:51.627 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds_log_expire": "1/5", 2023-01-24T18:32:51.627 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mds_migrator": "1/5", 2023-01-24T18:32:51.627 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_memdb": "4/5", 2023-01-24T18:32:51.628 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mgr": "2/5", 2023-01-24T18:32:51.628 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mgrc": "1/5", 2023-01-24T18:32:51.628 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_mon": "20/20", 2023-01-24T18:32:51.628 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_monc": "0/10", 2023-01-24T18:32:51.628 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_ms": "1/1", 2023-01-24T18:32:51.629 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_none": "0/5", 2023-01-24T18:32:51.629 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_objclass": "0/5", 2023-01-24T18:32:51.629 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_objectcacher": "0/5", 2023-01-24T18:32:51.629 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_objecter": "0/1", 2023-01-24T18:32:51.629 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_optracker": "0/5", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_osd": "1/5", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_paxos": "20/20", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_perfcounter": "1/5", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_prioritycache": "1/5", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rados": "0/5", 2023-01-24T18:32:51.630 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rbd": "0/5", 2023-01-24T18:32:51.631 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rbd_mirror": "0/5", 2023-01-24T18:32:51.631 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rbd_pwl": "0/5", 2023-01-24T18:32:51.631 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rbd_replay": "0/5", 2023-01-24T18:32:51.631 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_refs": "0/0", 2023-01-24T18:32:51.631 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_reserver": "1/1", 2023-01-24T18:32:51.632 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rgw": "1/5", 2023-01-24T18:32:51.632 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rgw_datacache": "1/5", 2023-01-24T18:32:51.632 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rgw_sync": "1/5", 2023-01-24T18:32:51.632 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_rocksdb": "4/5", 2023-01-24T18:32:51.632 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_cache": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_cleaner": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_device": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_journal": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_lba": "0/5", 2023-01-24T18:32:51.633 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_odata": "0/5", 2023-01-24T18:32:51.634 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_omap": "0/5", 2023-01-24T18:32:51.634 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_onode": "0/5", 2023-01-24T18:32:51.634 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_seastore_tm": "0/5", 2023-01-24T18:32:51.634 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_striper": "0/1", 2023-01-24T18:32:51.634 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_test": "0/5", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_throttle": "1/1", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_timer": "0/1", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "debug_tp": "0/5", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "device_failure_prediction_mode": "none", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-24T18:32:51.635 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-24T18:32:51.636 INFO:teuthology.orchestra.run.smithi169.stdout: "err_to_graylog": "false", 2023-01-24T18:32:51.636 INFO:teuthology.orchestra.run.smithi169.stdout: "err_to_journald": "false", 2023-01-24T18:32:51.636 INFO:teuthology.orchestra.run.smithi169.stdout: "err_to_stderr": "true", 2023-01-24T18:32:51.636 INFO:teuthology.orchestra.run.smithi169.stdout: "err_to_syslog": "false", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "event_tracing": "false", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_addr": "0.0.0.0", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_http_port": "9926", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_prio_limit": "5", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-24T18:32:51.637 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_sort_metrics": "true", 2023-01-24T18:32:51.638 INFO:teuthology.orchestra.run.smithi169.stdout: "exporter_stats_period": "5", 2023-01-24T18:32:51.638 INFO:teuthology.orchestra.run.smithi169.stdout: "fake_statfs_for_testing": "0", 2023-01-24T18:32:51.638 INFO:teuthology.orchestra.run.smithi169.stdout: "fatal_signal_handlers": "true", 2023-01-24T18:32:51.638 INFO:teuthology.orchestra.run.smithi169.stdout: "filer_max_purge_ops": "10", 2023-01-24T18:32:51.638 INFO:teuthology.orchestra.run.smithi169.stdout: "filer_max_truncate_ops": "128", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_apply_finisher_threads": "1", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_blackhole": "false", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_btrfs_clone_range": "true", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_btrfs_snap": "true", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_caller_concurrency": "10", 2023-01-24T18:32:51.639 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_collect_device_partition_information": "true", 2023-01-24T18:32:51.640 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_commit_timeout": "600.000000", 2023-01-24T18:32:51.640 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_debug_inject_read_err": "false", 2023-01-24T18:32:51.640 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_debug_omap_check": "false", 2023-01-24T18:32:51.640 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-24T18:32:51.640 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_debug_verify_split": "false", 2023-01-24T18:32:51.641 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_dump_file": "", 2023-01-24T18:32:51.641 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-24T18:32:51.641 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-24T18:32:51.641 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fadvise": "true", 2023-01-24T18:32:51.641 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fail_eio": "true", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fd_cache_shards": "16", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fd_cache_size": "128", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fiemap": "false", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fiemap_threshold": "4096", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-24T18:32:51.642 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-24T18:32:51.643 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_inject_stall": "0", 2023-01-24T18:32:51.643 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_journal_parallel": "false", 2023-01-24T18:32:51.643 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_journal_trailing": "false", 2023-01-24T18:32:51.643 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_journal_writeahead": "false", 2023-01-24T18:32:51.643 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_kill_at": "0", 2023-01-24T18:32:51.644 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-24T18:32:51.644 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-24T18:32:51.644 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-24T18:32:51.644 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-24T18:32:51.644 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-24T18:32:51.645 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattrs": "0", 2023-01-24T18:32:51.645 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-24T18:32:51.645 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-24T18:32:51.645 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-24T18:32:51.645 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-24T18:32:51.646 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_xattr_value_size": "0", 2023-01-24T18:32:51.646 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-24T18:32:51.646 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-24T18:32:51.646 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-24T18:32:51.646 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_merge_threshold": "-10", 2023-01-24T18:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-24T18:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_odsync_write": "false", 2023-01-24T18:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_omap_backend": "rocksdb", 2023-01-24T18:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_omap_backend_path": "", 2023-01-24T18:32:51.647 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-24T18:32:51.648 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-24T18:32:51.648 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-24T18:32:51.648 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_op_thread_timeout": "60", 2023-01-24T18:32:51.648 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_op_threads": "2", 2023-01-24T18:32:51.648 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_punch_hole": "false", 2023-01-24T18:32:51.649 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-24T18:32:51.649 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-24T18:32:51.649 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-24T18:32:51.649 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-24T18:32:51.649 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-24T18:32:51.650 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-24T18:32:51.650 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-24T18:32:51.650 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-24T18:32:51.650 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-24T18:32:51.650 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_queue_max_ops": "50", 2023-01-24T18:32:51.651 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-24T18:32:51.651 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_seek_data_hole": "false", 2023-01-24T18:32:51.651 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_sloppy_crc": "false", 2023-01-24T18:32:51.651 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-24T18:32:51.651 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_splice": "false", 2023-01-24T18:32:51.652 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_split_multiple": "2", 2023-01-24T18:32:51.652 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_split_rand_factor": "20", 2023-01-24T18:32:51.652 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_update_to": "1000", 2023-01-24T18:32:51.652 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-24T18:32:51.652 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_enable": "true", 2023-01-24T18:32:51.653 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-24T18:32:51.654 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-24T18:32:51.654 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-24T18:32:51.654 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-24T18:32:51.654 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-24T18:32:51.654 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-24T18:32:51.655 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_xfs_extsize": "false", 2023-01-24T18:32:51.655 INFO:teuthology.orchestra.run.smithi169.stdout: "filestore_zfs_snap": "false", 2023-01-24T18:32:51.655 INFO:teuthology.orchestra.run.smithi169.stdout: "fio_dir": "/tmp/fio", 2023-01-24T18:32:51.655 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "51ff070a-6967-4dc4-8c3c-d7750347282f", 2023-01-24T18:32:51.655 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_allow_other": "true", 2023-01-24T18:32:51.656 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_atomic_o_trunc": "true", 2023-01-24T18:32:51.656 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_big_writes": "true", 2023-01-24T18:32:51.656 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_debug": "false", 2023-01-24T18:32:51.656 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_default_permissions": "false", 2023-01-24T18:32:51.656 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_disable_pagecache": "false", 2023-01-24T18:32:51.657 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_max_write": "0", 2023-01-24T18:32:51.657 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_multithreaded": "true", 2023-01-24T18:32:51.657 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_require_active_mds": "true", 2023-01-24T18:32:51.657 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_set_user_groups": "true", 2023-01-24T18:32:51.657 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_splice_move": "true", 2023-01-24T18:32:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_splice_read": "true", 2023-01-24T18:32:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_splice_write": "true", 2023-01-24T18:32:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-24T18:32:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: "fuse_use_invalidate_cb": "true", 2023-01-24T18:32:51.658 INFO:teuthology.orchestra.run.smithi169.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-24T18:32:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: "gss_target_name": "ceph", 2023-01-24T18:32:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_file": "", 2023-01-24T18:32:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_inject_failure": "0", 2023-01-24T18:32:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_interval": "5", 2023-01-24T18:32:51.659 INFO:teuthology.orchestra.run.smithi169.stdout: "host": "smithi169", 2023-01-24T18:32:51.660 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-24T18:32:51.660 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-24T18:32:51.660 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-24T18:32:51.660 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-24T18:32:51.660 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-24T18:32:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-24T18:32:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-24T18:32:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-24T18:32:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-24T18:32:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-24T18:32:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-24T18:32:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-24T18:32:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-24T18:32:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: "inject_early_sigterm": "false", 2023-01-24T18:32:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: "jaeger_tracing_enable": "false", 2023-01-24T18:32:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_aio": "true", 2023-01-24T18:32:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_align_min_size": "65536", 2023-01-24T18:32:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_block_align": "true", 2023-01-24T18:32:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_block_size": "4096", 2023-01-24T18:32:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_dio": "true", 2023-01-24T18:32:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_discard": "false", 2023-01-24T18:32:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_force_aio": "false", 2023-01-24T18:32:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_ignore_corruption": "false", 2023-01-24T18:32:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_max_write_bytes": "10485760", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_max_write_entries": "100", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_replay_from": "0", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-24T18:32:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-24T18:32:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_write_header_frequency": "0", 2023-01-24T18:32:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: "journal_zero_on_create": "false", 2023-01-24T18:32:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: "journaler_prefetch_periods": "10", 2023-01-24T18:32:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: "journaler_prezero_periods": "5", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "journaler_write_head_interval": "15", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "keyfile": "", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_backend": "rocksdb", 2023-01-24T18:32:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_default_stripe_size": "65536", 2023-01-24T18:32:51.668 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_fsck_on_mount": "false", 2023-01-24T18:32:51.668 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-24T18:32:51.668 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_max_bytes": "67108864", 2023-01-24T18:32:51.668 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_max_ops": "512", 2023-01-24T18:32:51.669 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_nid_prealloc": "1024", 2023-01-24T18:32:51.669 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_onode_map_size": "1024", 2023-01-24T18:32:51.669 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-24T18:32:51.669 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_sync_submit_transaction": "false", 2023-01-24T18:32:51.669 INFO:teuthology.orchestra.run.smithi169.stdout: "kstore_sync_transaction": "false", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_block_size": "65536", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_bloom_size": "0", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_cache_size": "536870912", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_compact_on_mount": "false", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_compression": "false", 2023-01-24T18:32:51.670 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_log": "", 2023-01-24T18:32:51.671 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-24T18:32:51.671 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_max_open_files": "0", 2023-01-24T18:32:51.671 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_paranoid": "false", 2023-01-24T18:32:51.671 INFO:teuthology.orchestra.run.smithi169.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-24T18:32:51.671 INFO:teuthology.orchestra.run.smithi169.stdout: "librados_thread_count": "2", 2023-01-24T18:32:51.672 INFO:teuthology.orchestra.run.smithi169.stdout: "lockdep": "false", 2023-01-24T18:32:51.672 INFO:teuthology.orchestra.run.smithi169.stdout: "lockdep_force_backtrace": "false", 2023-01-24T18:32:51.672 INFO:teuthology.orchestra.run.smithi169.stdout: "log_coarse_timestamps": "true", 2023-01-24T18:32:51.672 INFO:teuthology.orchestra.run.smithi169.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-24T18:32:51.672 INFO:teuthology.orchestra.run.smithi169.stdout: "log_flush_on_exit": "false", 2023-01-24T18:32:51.673 INFO:teuthology.orchestra.run.smithi169.stdout: "log_graylog_host": "127.0.0.1", 2023-01-24T18:32:51.673 INFO:teuthology.orchestra.run.smithi169.stdout: "log_graylog_port": "12201", 2023-01-24T18:32:51.673 INFO:teuthology.orchestra.run.smithi169.stdout: "log_max_new": "1000", 2023-01-24T18:32:51.673 INFO:teuthology.orchestra.run.smithi169.stdout: "log_max_recent": "10000", 2023-01-24T18:32:51.673 INFO:teuthology.orchestra.run.smithi169.stdout: "log_stderr_prefix": "", 2023-01-24T18:32:51.674 INFO:teuthology.orchestra.run.smithi169.stdout: "log_stop_at_utilization": "0.970000", 2023-01-24T18:32:51.674 INFO:teuthology.orchestra.run.smithi169.stdout: "log_to_file": "true", 2023-01-24T18:32:51.674 INFO:teuthology.orchestra.run.smithi169.stdout: "log_to_graylog": "false", 2023-01-24T18:32:51.674 INFO:teuthology.orchestra.run.smithi169.stdout: "log_to_journald": "false", 2023-01-24T18:32:51.674 INFO:teuthology.orchestra.run.smithi169.stdout: "log_to_stderr": "false", 2023-01-24T18:32:51.675 INFO:teuthology.orchestra.run.smithi169.stdout: "log_to_syslog": "false", 2023-01-24T18:32:51.675 INFO:teuthology.orchestra.run.smithi169.stdout: "max_rotating_auth_attempts": "10", 2023-01-24T18:32:51.675 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_action_on_write_error": "1", 2023-01-24T18:32:51.675 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_alternate_name_max": "8192", 2023-01-24T18:32:51.675 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_asio_thread_count": "2", 2023-01-24T18:32:51.676 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_export_pin": "true", 2023-01-24T18:32:51.676 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_fragment_dirs": "true", 2023-01-24T18:32:51.676 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-24T18:32:51.676 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_fragment_interval": "5", 2023-01-24T18:32:51.676 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-24T18:32:51.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-24T18:32:51.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_interval": "10", 2023-01-24T18:32:51.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_max": "-1", 2023-01-24T18:32:51.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_max_until": "-1", 2023-01-24T18:32:51.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_merge_size": "50", 2023-01-24T18:32:51.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_midchunk": "0.300000", 2023-01-24T18:32:51.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-24T18:32:51.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_min_start": "0.200000", 2023-01-24T18:32:51.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_minchunk": "0.001000", 2023-01-24T18:32:51.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_mode": "0", 2023-01-24T18:32:51.679 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_need_max": "1.200000", 2023-01-24T18:32:51.679 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_need_min": "0.800000", 2023-01-24T18:32:51.679 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-24T18:32:51.679 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-24T18:32:51.679 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_split_bits": "3", 2023-01-24T18:32:51.680 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-24T18:32:51.680 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_split_size": "10000", 2023-01-24T18:32:51.680 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-24T18:32:51.680 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_target_decay": "10.000000", 2023-01-24T18:32:51.680 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-24T18:32:51.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_beacon_grace": "15.000000", 2023-01-24T18:32:51.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_beacon_interval": "4.000000", 2023-01-24T18:32:51.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-24T18:32:51.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-24T18:32:51.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_mid": "0.700000", 2023-01-24T18:32:51.682 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_release_free_interval": "10", 2023-01-24T18:32:51.682 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_reservation": "0.050000", 2023-01-24T18:32:51.682 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-24T18:32:51.682 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_trim_interval": "1", 2023-01-24T18:32:51.682 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache_trim_threshold": "262144", 2023-01-24T18:32:51.683 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-24T18:32:51.683 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-24T18:32:51.683 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-24T18:32:51.683 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_client_prealloc_inos": "1000", 2023-01-24T18:32:51.683 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-24T18:32:51.684 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_connect_bootstrapping": "false", 2023-01-24T18:32:51.684 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_damage_table_max_entries": "10000", 2023-01-24T18:32:51.684 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-24T18:32:51.684 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_debug_auth_pins": "false", 2023-01-24T18:32:51.684 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_debug_frag": "false", 2023-01-24T18:32:51.685 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_debug_scatterstat": "false", 2023-01-24T18:32:51.685 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_debug_subtrees": "false", 2023-01-24T18:32:51.685 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_decay_halflife": "5.000000", 2023-01-24T18:32:51.685 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_default_dir_hash": "2", 2023-01-24T18:32:51.685 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_defer_session_stale": "true", 2023-01-24T18:32:51.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_deny_all_reconnect": "false", 2023-01-24T18:32:51.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dir_keys_per_op": "16384", 2023-01-24T18:32:51.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dir_max_commit_size": "10", 2023-01-24T18:32:51.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dir_max_entries": "0", 2023-01-24T18:32:51.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dump_cache_on_map": "false", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_early_reply": "true", 2023-01-24T18:32:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_enable_op_tracker": "true", 2023-01-24T18:32:51.688 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_enforce_unique_name": "true", 2023-01-24T18:32:51.688 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-24T18:32:51.688 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-24T18:32:51.688 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_export_ephemeral_random": "true", 2023-01-24T18:32:51.688 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_health_summarize_threshold": "10", 2023-01-24T18:32:51.689 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-24T18:32:51.690 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-24T18:32:51.690 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_inject_health_dummy": "false", 2023-01-24T18:32:51.690 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_inject_migrator_session_race": "false", 2023-01-24T18:32:51.690 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-24T18:32:51.690 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_join_fs": "", 2023-01-24T18:32:51.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_journal_format": "1", 2023-01-24T18:32:51.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_create_at": "0", 2023-01-24T18:32:51.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_export_at": "0", 2023-01-24T18:32:51.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_import_at": "0", 2023-01-24T18:32:51.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_journal_at": "0", 2023-01-24T18:32:51.692 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_journal_expire_at": "0", 2023-01-24T18:32:51.692 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_journal_replay_at": "0", 2023-01-24T18:32:51.692 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_link_at": "0", 2023-01-24T18:32:51.692 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_mdstable_at": "0", 2023-01-24T18:32:51.692 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_openc_at": "0", 2023-01-24T18:32:51.693 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_kill_rename_at": "0", 2023-01-24T18:32:51.693 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_events_per_segment": "1024", 2023-01-24T18:32:51.693 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_max_events": "-1", 2023-01-24T18:32:51.693 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_max_segments": "128", 2023-01-24T18:32:51.693 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_pause": "false", 2023-01-24T18:32:51.694 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_segment_size": "0", 2023-01-24T18:32:51.694 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-24T18:32:51.694 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log_warn_factor": "2.000000", 2023-01-24T18:32:51.694 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_caps_per_client": "1048576", 2023-01-24T18:32:51.694 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_completed_flushes": "100000", 2023-01-24T18:32:51.695 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_completed_requests": "100000", 2023-01-24T18:32:51.695 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_export_size": "20971520", 2023-01-24T18:32:51.695 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_file_recover": "32", 2023-01-24T18:32:51.695 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_purge_files": "64", 2023-01-24T18:32:51.695 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_purge_ops": "8192", 2023-01-24T18:32:51.696 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-24T18:32:51.696 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-24T18:32:51.696 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-24T18:32:51.696 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_snaps_per_dir": "100", 2023-01-24T18:32:51.696 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_metrics_update_interval": "2", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_min_caps_per_client": "100", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_min_caps_working_set": "10000", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_numa_node": "-1", 2023-01-24T18:32:51.697 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-24T18:32:51.698 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_op_complaint_time": "30.000000", 2023-01-24T18:32:51.698 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_op_history_duration": "600", 2023-01-24T18:32:51.698 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_op_history_size": "20", 2023-01-24T18:32:51.698 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_op_log_threshold": "5", 2023-01-24T18:32:51.698 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_ping_grace": "15", 2023-01-24T18:32:51.699 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_ping_interval": "5", 2023-01-24T18:32:51.699 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-24T18:32:51.699 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-24T18:32:51.699 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_max_caps": "30000", 2023-01-24T18:32:51.699 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-24T18:32:51.700 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-24T18:32:51.700 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-24T18:32:51.700 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_recall_warning_threshold": "262144", 2023-01-24T18:32:51.700 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-24T18:32:51.700 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_replay_interval": "1.000000", 2023-01-24T18:32:51.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-24T18:32:51.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-24T18:32:51.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_root_ino_gid": "0", 2023-01-24T18:32:51.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_root_ino_uid": "0", 2023-01-24T18:32:51.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-24T18:32:51.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-24T18:32:51.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-24T18:32:51.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-24T18:32:51.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-24T18:32:51.703 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-24T18:32:51.703 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-24T18:32:51.703 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-24T18:32:51.703 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-24T18:32:51.703 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_shutdown_check": "0", 2023-01-24T18:32:51.704 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_skip_ino": "0", 2023-01-24T18:32:51.704 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-24T18:32:51.704 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_snap_max_uid": "4294967294", 2023-01-24T18:32:51.704 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_snap_min_uid": "0", 2023-01-24T18:32:51.704 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_snap_rstat": "false", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_standby_replay_damaged": "false", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_symlink_recovery": "true", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_thrash_exports": "0", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_thrash_fragments": "0", 2023-01-24T18:32:51.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_tick_interval": "5.000000", 2023-01-24T18:32:51.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_valgrind_exit": "false", 2023-01-24T18:32:51.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_verify_backtrace": "1", 2023-01-24T18:32:51.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_verify_scatter": "false", 2023-01-24T18:32:51.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-24T18:32:51.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_wipe_sessions": "false", 2023-01-24T18:32:51.707 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool_debug": "false", 2023-01-24T18:32:51.707 INFO:teuthology.orchestra.run.smithi169.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-24T18:32:51.707 INFO:teuthology.orchestra.run.smithi169.stdout: "memstore_device_bytes": "1073741824", 2023-01-24T18:32:51.707 INFO:teuthology.orchestra.run.smithi169.stdout: "memstore_page_set": "false", 2023-01-24T18:32:51.707 INFO:teuthology.orchestra.run.smithi169.stdout: "memstore_page_size": "65536", 2023-01-24T18:32:51.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_client_bytes": "134217728", 2023-01-24T18:32:51.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_client_messages": "512", 2023-01-24T18:32:51.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-24T18:32:51.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-24T18:32:51.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-24T18:32:51.709 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-24T18:32:51.709 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_disabled_modules": "", 2023-01-24T18:32:51.709 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-24T18:32:51.709 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_max_pg_num_change": "128", 2023-01-24T18:32:51.709 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_mds_bytes": "134217728", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_mds_messages": "128", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_mon_bytes": "134217728", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_mon_messages": "128", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_osd_bytes": "536870912", 2023-01-24T18:32:51.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_osd_messages": "8192", 2023-01-24T18:32:51.711 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_pool": "true", 2023-01-24T18:32:51.711 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-24T18:32:51.711 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_standby_modules": "true", 2023-01-24T18:32:51.711 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_stats_period": "5", 2023-01-24T18:32:51.711 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_stats_threshold": "5", 2023-01-24T18:32:51.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_tick_period": "2", 2023-01-24T18:32:51.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-24T18:32:51.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-24T18:32:51.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_allow_pool_delete": "true", 2023-01-24T18:32:51.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_allow_pool_size_one": "true", 2023-01-24T18:32:51.713 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_auth_validate_all_caps": "true", 2023-01-24T18:32:51.713 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-24T18:32:51.713 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-24T18:32:51.713 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_bytes": "104857600", 2023-01-24T18:32:51.713 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_directed_command_retry": "5", 2023-01-24T18:32:51.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-24T18:32:51.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-24T18:32:51.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-24T18:32:51.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-24T18:32:51.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_hunt_parallel": "3", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_log_interval": "1.000000", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_ping_interval": "10.000000", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-24T18:32:51.715 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-24T18:32:51.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-24T18:32:51.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_file_level": "debug", 2023-01-24T18:32:51.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_file": "true", 2023-01-24T18:32:51.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-24T18:32:51.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-24T18:32:51.717 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-24T18:32:51.717 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_journald": "false", 2023-01-24T18:32:51.717 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-24T18:32:51.717 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-24T18:32:51.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-24T18:32:51.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-24T18:32:51.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_compact_on_bootstrap": "false", 2023-01-24T18:32:51.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_compact_on_start": "false", 2023-01-24T18:32:51.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_compact_on_trim": "true", 2023-01-24T18:32:51.719 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-24T18:32:51.719 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-24T18:32:51.719 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-24T18:32:51.719 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_cpu_threads": "4", 2023-01-24T18:32:51.719 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_crush_min_required_version": "hammer", 2023-01-24T18:32:51.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_daemon_bytes": "419430400", 2023-01-24T18:32:51.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-24T18:32:51.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_data_avail_crit": "5", 2023-01-24T18:32:51.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_data_avail_warn": "5", 2023-01-24T18:32:51.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_data_size_warn": "16106127360", 2023-01-24T18:32:51.721 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-24T18:32:51.721 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-24T18:32:51.721 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_dump_json": "false", 2023-01-24T18:32:51.721 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-24T18:32:51.721 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_dump_transactions": "false", 2023-01-24T18:32:51.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_extra_checks": "false", 2023-01-24T18:32:51.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-24T18:32:51.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-24T18:32:51.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_no_require_pacific": "false", 2023-01-24T18:32:51.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_no_require_quincy": "false", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_down_mkfs_grace": "120", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_election_default_strategy": "3", 2023-01-24T18:32:51.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_election_timeout": "5.000000", 2023-01-24T18:32:51.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-24T18:32:51.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_elector_ping_divisor": "2", 2023-01-24T18:32:51.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-24T18:32:51.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_enable_op_tracker": "true", 2023-01-24T18:32:51.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_fake_pool_delete": "false", 2023-01-24T18:32:51.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_force_quorum_join": "false", 2023-01-24T18:32:51.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_globalid_prealloc": "10000", 2023-01-24T18:32:51.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_detail_to_clog": "false", 2023-01-24T18:32:51.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_log_update_period": "5", 2023-01-24T18:32:51.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_max_detail": "50", 2023-01-24T18:32:51.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_to_clog": "true", 2023-01-24T18:32:51.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_to_clog_interval": "600", 2023-01-24T18:32:51.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-24T18:32:51.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_host": "172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790],172.21.15.186", 2023-01-24T18:32:51.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_host_override": "", 2023-01-24T18:32:51.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_initial_members": "", 2023-01-24T18:32:51.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-24T18:32:51.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-24T18:32:51.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-24T18:32:51.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-24T18:32:51.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-24T18:32:51.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_lease": "5.000000", 2023-01-24T18:32:51.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-24T18:32:51.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-24T18:32:51.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_log_full_interval": "50", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_log_max": "10000", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_log_max_summary": "50", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_log_epochs": "500", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-24T18:32:51.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-24T18:32:51.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_osd": "10000", 2023-01-24T18:32:51.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_pg_per_osd": "10000", 2023-01-24T18:32:51.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_pool_pg_num": "65536", 2023-01-24T18:32:51.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-24T18:32:51.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-24T18:32:51.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mds_force_trim_to": "0", 2023-01-24T18:32:51.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mds_skip_sanity": "false", 2023-01-24T18:32:51.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_memory_autotune": "true", 2023-01-24T18:32:51.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_memory_target": "2147483648", 2023-01-24T18:32:51.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_beacon_grace": "30", 2023-01-24T18:32:51.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-24T18:32:51.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_digest_period": "5", 2023-01-24T18:32:51.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_inactive_grace": "60", 2023-01-24T18:32:51.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-24T18:32:51.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_min_osdmap_epochs": "50", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_complaint_time": "30", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_history_duration": "600", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_history_size": "20", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_history_slow_op_size": "20", 2023-01-24T18:32:51.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_op_log_threshold": "5", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_auto_mark_in": "false", 2023-01-24T18:32:51.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-24T18:32:51.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-24T18:32:51.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-24T18:32:51.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_cache_size": "500", 2023-01-24T18:32:51.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-24T18:32:51.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-24T18:32:51.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-24T18:32:51.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_down_out_interval": "600", 2023-01-24T18:32:51.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-24T18:32:51.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-24T18:32:51.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_force_trim_to": "0", 2023-01-24T18:32:51.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-24T18:32:51.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-24T18:32:51.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-24T18:32:51.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-24T18:32:51.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-24T18:32:51.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-24T18:32:51.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-24T18:32:51.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-24T18:32:51.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_min_down_reporters": "2", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-24T18:32:51.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-24T18:32:51.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_report_timeout": "900", 2023-01-24T18:32:51.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-24T18:32:51.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-24T18:32:51.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-24T18:32:51.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-24T18:32:51.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_stuck_threshold": "60", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-24T18:32:51.742 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-24T18:32:51.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_probe_timeout": "2.000000", 2023-01-24T18:32:51.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_reweight_max_change": "0.050000", 2023-01-24T18:32:51.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_reweight_max_osds": "4", 2023-01-24T18:32:51.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-24T18:32:51.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-24T18:32:51.744 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-24T18:32:51.744 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-24T18:32:51.744 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-24T18:32:51.744 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_scrub_interval": "300", 2023-01-24T18:32:51.744 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_scrub_max_keys": "100", 2023-01-24T18:32:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_scrub_timeout": "300", 2023-01-24T18:32:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_session_timeout": "300", 2023-01-24T18:32:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_smart_report_timeout": "5", 2023-01-24T18:32:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_stat_smooth_intervals": "6", 2023-01-24T18:32:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-24T18:32:51.746 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_stretch_pool_min_size": "2", 2023-01-24T18:32:51.746 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_stretch_pool_size": "4", 2023-01-24T18:32:51.746 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-24T18:32:51.746 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-24T18:32:51.746 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_debug": "false", 2023-01-24T18:32:51.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-24T18:32:51.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-24T18:32:51.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_provider_kill_at": "0", 2023-01-24T18:32:51.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_requester_kill_at": "0", 2023-01-24T18:32:51.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_sync_timeout": "60.000000", 2023-01-24T18:32:51.748 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_target_pg_per_osd": "100", 2023-01-24T18:32:51.748 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_tick_interval": "5", 2023-01-24T18:32:51.748 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_timecheck_interval": "300.000000", 2023-01-24T18:32:51.748 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-24T18:32:51.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_older_version_delay": "604800", 2023-01-24T18:32:51.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-24T18:32:51.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-24T18:32:51.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-24T18:32:51.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-24T18:32:51.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-24T18:32:51.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-24T18:32:51.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-24T18:32:51.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_misplaced": "false", 2023-01-24T18:32:51.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-24T18:32:51.751 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_older_version": "true", 2023-01-24T18:32:51.751 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-24T18:32:51.751 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-24T18:32:51.751 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-24T18:32:51.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-24T18:32:51.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-24T18:32:51.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-24T18:32:51.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-24T18:32:51.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-24T18:32:51.753 INFO:teuthology.orchestra.run.smithi169.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-24T18:32:51.753 INFO:teuthology.orchestra.run.smithi169.stdout: "monmap": "", 2023-01-24T18:32:51.753 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_op_threads": "3", 2023-01-24T18:32:51.753 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-24T18:32:51.753 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_cm": "false", 2023-01-24T18:32:51.754 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_device_name": "", 2023-01-24T18:32:51.754 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_dscp": "96", 2023-01-24T18:32:51.754 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-24T18:32:51.754 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-24T18:32:51.754 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_local_gid": "", 2023-01-24T18:32:51.755 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-24T18:32:51.755 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_port_num": "1", 2023-01-24T18:32:51.755 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-24T18:32:51.755 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-24T18:32:51.756 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_roce_ver": "1", 2023-01-24T18:32:51.756 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_send_buffers": "1024", 2023-01-24T18:32:51.756 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_sl": "3", 2023-01-24T18:32:51.756 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_support_srq": "true", 2023-01-24T18:32:51.756 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_rdma_type": "ib", 2023-01-24T18:32:51.757 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_async_reap_threshold": "5", 2023-01-24T18:32:51.757 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_before_connect": "false", 2023-01-24T18:32:51.757 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_ipv4": "true", 2023-01-24T18:32:51.757 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_ipv6": "false", 2023-01-24T18:32:51.757 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_msgr1": "false", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_msgr2": "true", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_port_max": "7568", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_port_min": "6800", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_prefer_ipv4": "false", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_retry_count": "3", 2023-01-24T18:32:51.758 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_bind_retry_delay": "5", 2023-01-24T18:32:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_blackhole_client": "false", 2023-01-24T18:32:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_blackhole_mds": "false", 2023-01-24T18:32:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_blackhole_mgr": "false", 2023-01-24T18:32:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_blackhole_mon": "false", 2023-01-24T18:32:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_blackhole_osd": "false", 2023-01-24T18:32:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_client_mode": "crc secure", 2023-01-24T18:32:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_cluster_mode": "crc secure", 2023-01-24T18:32:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_cluster_type": "", 2023-01-24T18:32:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_compress_secure": "false", 2023-01-24T18:32:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_connection_idle_timeout": "900", 2023-01-24T18:32:51.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_connection_ready_timeout": "10", 2023-01-24T18:32:51.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_crc_data": "true", 2023-01-24T18:32:51.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_crc_header": "true", 2023-01-24T18:32:51.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_die_on_bad_msg": "false", 2023-01-24T18:32:51.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_die_on_bug": "true", 2023-01-24T18:32:51.762 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_die_on_old_message": "true", 2023-01-24T18:32:51.762 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_die_on_skipped_message": "false", 2023-01-24T18:32:51.762 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_die_on_unhandled_msg": "false", 2023-01-24T18:32:51.762 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-01-24T18:32:51.762 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_coremask": "0xF", 2023-01-24T18:32:51.763 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-01-24T18:32:51.763 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_devs_allowlist": "", 2023-01-24T18:32:51.763 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_enable_tso": "true", 2023-01-24T18:32:51.763 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-01-24T18:32:51.763 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-01-24T18:32:51.764 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_hugepages": "", 2023-01-24T18:32:51.764 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_hw_flow_control": "true", 2023-01-24T18:32:51.764 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-01-24T18:32:51.764 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_lro": "true", 2023-01-24T18:32:51.764 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_memory_channel": "4", 2023-01-24T18:32:51.765 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-01-24T18:32:51.765 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_pmd": "", 2023-01-24T18:32:51.765 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_port_id": "0", 2023-01-24T18:32:51.765 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-01-24T18:32:51.765 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dump_corrupt_message_level": "1", 2023-01-24T18:32:51.766 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_dump_on_send": "false", 2023-01-24T18:32:51.766 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_initial_backoff": "0.200000", 2023-01-24T18:32:51.766 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_inject_delay_max": "1.000000", 2023-01-24T18:32:51.766 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_inject_delay_probability": "0.005000", 2023-01-24T18:32:51.766 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_inject_delay_type": "osd", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_inject_internal_delays": "0.002000", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_inject_socket_failures": "2500", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_learn_addr_from_peer": "true", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_max_accept_failures": "4", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_max_backoff": "15.000000", 2023-01-24T18:32:51.767 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_mon_client_mode": "secure crc", 2023-01-24T18:32:51.768 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_mon_cluster_mode": "secure crc", 2023-01-24T18:32:51.768 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_mon_service_mode": "secure crc", 2023-01-24T18:32:51.768 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_osd_compress_min_size": "1024", 2023-01-24T18:32:51.768 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_osd_compress_mode": "none", 2023-01-24T18:32:51.768 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_pq_min_cost": "65536", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_public_type": "", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_service_mode": "crc secure", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_tcp_listen_backlog": "512", 2023-01-24T18:32:51.769 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_tcp_nodelay": "true", 2023-01-24T18:32:51.770 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-01-24T18:32:51.770 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_tcp_rcvbuf": "0", 2023-01-24T18:32:51.770 INFO:teuthology.orchestra.run.smithi169.stdout: "ms_type": "async", 2023-01-24T18:32:51.770 INFO:teuthology.orchestra.run.smithi169.stdout: "no_config_file": "false", 2023-01-24T18:32:51.770 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_completion_locks_per_session": "32", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_debug_inject_relock_delay": "false", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_inflight_op_bytes": "104857600", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_inflight_ops": "1024", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_inject_no_watch_ping": "false", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-01-24T18:32:51.771 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_tick_interval": "5.000000", 2023-01-24T18:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter_timeout": "10.000000", 2023-01-24T18:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: "objectstore_blackhole": "false", 2023-01-24T18:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: "openssl_engine_opts": "", 2023-01-24T18:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_delay_time": "5.000000", 2023-01-24T18:32:51.772 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_hist_halflife": "1000", 2023-01-24T18:32:51.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_max_low_ops": "2", 2023-01-24T18:32:51.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_max_ops": "4", 2023-01-24T18:32:51.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-01-24T18:32:51.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_quantize_effort": "0.100000", 2023-01-24T18:32:51.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_agent_slop": "0.020000", 2023-01-24T18:32:51.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-01-24T18:32:51.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_allow_recovery_below_min_size": "true", 2023-01-24T18:32:51.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_asio_thread_count": "2", 2023-01-24T18:32:51.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_async_recovery_min_cost": "100", 2023-01-24T18:32:51.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_auto_mark_unfound_lost": "false", 2023-01-24T18:32:51.775 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backfill_retry_interval": "30.000000", 2023-01-24T18:32:51.775 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backfill_scan_max": "512", 2023-01-24T18:32:51.775 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backfill_scan_min": "64", 2023-01-24T18:32:51.775 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backoff_on_degraded": "false", 2023-01-24T18:32:51.775 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backoff_on_peering": "false", 2023-01-24T18:32:51.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_backoff_on_unfound": "true", 2023-01-24T18:32:51.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_beacon_report_interval": "300", 2023-01-24T18:32:51.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bench_duration": "30", 2023-01-24T18:32:51.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-01-24T18:32:51.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bench_max_block_size": "67108864", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bench_small_size_max_iops": "100", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_blkin_trace_all": "false", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_check_for_log_corruption": "false", 2023-01-24T18:32:51.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-01-24T18:32:51.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T18:32:51.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-01-24T18:32:51.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T18:32:51.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_class_update_on_start": "true", 2023-01-24T18:32:51.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_client_message_cap": "256", 2023-01-24T18:32:51.779 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_client_message_size_cap": "524288000", 2023-01-24T18:32:51.779 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_client_op_priority": "63", 2023-01-24T18:32:51.779 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_client_watch_timeout": "30", 2023-01-24T18:32:51.779 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_command_max_records": "256", 2023-01-24T18:32:51.779 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_compact_on_start": "false", 2023-01-24T18:32:51.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-01-24T18:32:51.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_crush_chooseleaf_type": "0", 2023-01-24T18:32:51.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_crush_initial_weight": "-1.000000", 2023-01-24T18:32:51.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_crush_update_on_start": "true", 2023-01-24T18:32:51.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_crush_update_weight_set": "true", 2023-01-24T18:32:51.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-01-24T18:32:51.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-01-24T18:32:51.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-01-24T18:32:51.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_drop_ping_duration": "0", 2023-01-24T18:32:51.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-01-24T18:32:51.782 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_feed_pullee": "-1", 2023-01-24T18:32:51.782 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-01-24T18:32:51.782 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-01-24T18:32:51.782 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-01-24T18:32:51.782 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_misdirected_ops": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_no_acting_change": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_no_purge_strays": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_op_order": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_pg_log_writeout": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_pretend_recovery_active": "false", 2023-01-24T18:32:51.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-01-24T18:32:51.784 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-01-24T18:32:51.784 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_shutdown": "false", 2023-01-24T18:32:51.784 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-01-24T18:32:51.784 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-01-24T18:32:51.784 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_verify_cached_snaps": "false", 2023-01-24T18:32:51.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_verify_missing_on_start": "false", 2023-01-24T18:32:51.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_verify_snaps": "false", 2023-01-24T18:32:51.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-01-24T18:32:51.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-01-24T18:32:51.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_keys": "1024", 2023-01-24T18:32:51.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-01-24T18:32:51.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-01-24T18:32:51.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-01-24T18:32:51.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_stride": "524288", 2023-01-24T18:32:51.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-01-24T18:32:51.787 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_default_data_pool_replay_window": "5", 2023-01-24T18:32:51.787 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_default_notify_timeout": "30", 2023-01-24T18:32:51.787 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_delete_sleep": "0.000000", 2023-01-24T18:32:51.787 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-01-24T18:32:51.787 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-01-24T18:32:51.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-01-24T18:32:51.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_discard_disconnected_ops": "true", 2023-01-24T18:32:51.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_enable_op_tracker": "true", 2023-01-24T18:32:51.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-01-24T18:32:51.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-01-24T18:32:51.789 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-01-24T18:32:51.789 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_fast_info": "true", 2023-01-24T18:32:51.789 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_fast_shutdown": "true", 2023-01-24T18:32:51.789 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_fast_shutdown_timeout": "15", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_function_tracing": "false", 2023-01-24T18:32:51.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_grace": "20", 2023-01-24T18:32:51.791 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_interval": "6", 2023-01-24T18:32:51.791 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-01-24T18:32:51.791 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_min_peers": "10", 2023-01-24T18:32:51.791 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_min_size": "2000", 2023-01-24T18:32:51.791 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_heartbeat_stale": "600", 2023-01-24T18:32:51.792 INFO:teuthology.orc